[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:34:36 dist-test-slave-3kk6 krb5kdc[31279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:34:36 dist-test-slave-3kk6 krb5kdc[31279](info): set up 2 sockets
Aug 09 05:34:36 dist-test-slave-3kk6 krb5kdc[31279](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 05:34:37.229210 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:34:37 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:34:37.420925 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35941
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:37.766906 31299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:37.767223 31299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:37.767315 31299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:37.774891 31299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:34:37.775024 31299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:37.775074 31299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:37.775135 31299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:34:37.775182 31299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:34:37.785406 31299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:37.786882 31299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:37.788244 31299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:37.800663 31307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:37.800767 31312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:37.800705 31309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:37.801558 31299 server_base.cc:1034] running on GCE node
I20260809 05:34:37.802404 31299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:37.804025 31299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:37.805294 31299 hybrid_clock.cc:648] HybridClock initialized: now 1786253677805258 us; error 105 us; skew 500 ppm
I20260809 05:34:37.805671 31299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:37 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253677, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:37.809015 31299 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:34:37.811208 31299 webserver.cc:460] Webserver started at http://127.30.137.254:44419/ using document root <none> and password file <none>
I20260809 05:34:37.811985 31299 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:37.812140 31299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:37.812487 31299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:37.815757 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "acc98fcd15c143f396c6f26ac0beb655"
format_stamp: "Formatted at 2026-08-09 05:34:37 on dist-test-slave-3kk6"
server_key: "be710444e45fc112a5c11c018104d000"
server_key_iv: "907e6f94ee4c4daa9fbd499c8d2efff9"
server_key_version: "encryptionkey@0"
I20260809 05:34:37.816578 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "acc98fcd15c143f396c6f26ac0beb655"
format_stamp: "Formatted at 2026-08-09 05:34:37 on dist-test-slave-3kk6"
server_key: "be710444e45fc112a5c11c018104d000"
server_key_iv: "907e6f94ee4c4daa9fbd499c8d2efff9"
server_key_version: "encryptionkey@0"
I20260809 05:34:37.821933 31299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 05:34:37.825475 31322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:37.826869 31299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:34:37.827065 31299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "acc98fcd15c143f396c6f26ac0beb655"
format_stamp: "Formatted at 2026-08-09 05:34:37 on dist-test-slave-3kk6"
server_key: "be710444e45fc112a5c11c018104d000"
server_key_iv: "907e6f94ee4c4daa9fbd499c8d2efff9"
server_key_version: "encryptionkey@0"
I20260809 05:34:37.827271 31299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:37.844456 31299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:37.848610 31299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:37.848928 31299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:37.868886 31299 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35941
I20260809 05:34:37.868912 31398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35941 every 8 connection(s)
I20260809 05:34:37.870373 31299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:34:37.871356 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31299
I20260809 05:34:37.871740 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:34:37.873679 31271 external_mini_cluster.cc:1428] Setting key 945b2e6ece75eb388feb362bab2efa2a
I20260809 05:34:37.874593 31399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:37.886736 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655: Bootstrap starting.
I20260809 05:34:37.890209 31399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:37.891397 31399 log.cc:824] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:37.894402 31399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655: No bootstrap required, opened a new log
Aug 09 05:34:37 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:37.901918 31399 raft_consensus.cc:348] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acc98fcd15c143f396c6f26ac0beb655" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35941 } }
I20260809 05:34:37.902421 31399 raft_consensus.cc:374] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:37.902540 31399 raft_consensus.cc:729] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acc98fcd15c143f396c6f26ac0beb655, State: Initialized, Role: FOLLOWER
I20260809 05:34:37.903501 31399 consensus_queue.cc:260] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acc98fcd15c143f396c6f26ac0beb655" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35941 } }
I20260809 05:34:37.903808 31399 raft_consensus.cc:388] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:34:37.903975 31399 raft_consensus.cc:482] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:34:37.904227 31399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:37.907619 31399 raft_consensus.cc:504] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acc98fcd15c143f396c6f26ac0beb655" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35941 } }
I20260809 05:34:37.908247 31399 leader_election.cc:302] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acc98fcd15c143f396c6f26ac0beb655; no voters:
I20260809 05:34:37.908999 31399 leader_election.cc:288] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:34:37.909127 31406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:37.911713 31399 sys_catalog.cc:564] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:34:37.915704 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:37.877766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.30.137.254:35941)
0809 05:34:37.878655 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:37.878694 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:37.879925 (+ 1231us) server_negotiation.cc:407] Connection header received
0809 05:34:37.889006 (+ 9081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:37.889027 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:37.889423 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:37.889811 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:37.893328 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:37.894790 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:37.896531 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:37.897246 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:37.897481 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:37.904102 (+ 6621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:37.904120 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:37.904144 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:37.904217 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:37.908239 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:37.909538 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:37.909546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:37.909553 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:37.909902 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:37.910532 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:37.910537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:37.910540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:37.911360 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:37.911691 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:37.913223 (+ 1532us) server_negotiation.cc:299] Negotiation successful
0809 05:34:37.913809 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":434,"threads_started":1}
I20260809 05:34:37.916394 31406 raft_consensus.cc:686] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [term 1 LEADER]: Becoming Leader. State: Replica: acc98fcd15c143f396c6f26ac0beb655, State: Running, Role: LEADER
I20260809 05:34:37.917009 31406 consensus_queue.cc:237] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acc98fcd15c143f396c6f26ac0beb655" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35941 } }
I20260809 05:34:37.925812 31407 sys_catalog.cc:455] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acc98fcd15c143f396c6f26ac0beb655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acc98fcd15c143f396c6f26ac0beb655" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35941 } } }
I20260809 05:34:37.926369 31407 sys_catalog.cc:458] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:37.927222 31408 sys_catalog.cc:455] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acc98fcd15c143f396c6f26ac0beb655. Latest consensus state: current_term: 1 leader_uuid: "acc98fcd15c143f396c6f26ac0beb655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acc98fcd15c143f396c6f26ac0beb655" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35941 } } }
I20260809 05:34:37.927887 31408 sys_catalog.cc:458] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655 [sys.catalog]: This master's current role is: LEADER
W20260809 05:34:37.933910 31430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:34:37.934047 31430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:34:37.934204 31432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:34:37.935966 31432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:34:37.945351 31432 catalog_manager.cc:1347] Generated new cluster ID: 8a0bd995b314428d854104fe6cc8a350
I20260809 05:34:37.945485 31432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:34:37.986400 31432 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:34:37.987793 31432 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:34:37.998584 31432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acc98fcd15c143f396c6f26ac0beb655: Generated new TSK 0
I20260809 05:34:37.999565 31432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:34:38.058706 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35941
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:38.420025 31443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:38.420404 31443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:38.420583 31443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:38.428566 31443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:38.428742 31443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:38.428947 31443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:34:38.439626 31443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:38.441061 31443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:38.442600 31443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:38.456888 31453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:38.456939 31454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:38.456955 31460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:38.457744 31443 server_base.cc:1034] running on GCE node
I20260809 05:34:38.458425 31443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:38.459620 31443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:38.460866 31443 hybrid_clock.cc:648] HybridClock initialized: now 1786253678460813 us; error 49 us; skew 500 ppm
I20260809 05:34:38.461218 31443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:38 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253678, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:38.464641 31443 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:34:38.466216 31443 webserver.cc:460] Webserver started at http://127.30.137.193:43763/ using document root <none> and password file <none>
I20260809 05:34:38.467032 31443 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:38.467190 31443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:38.467538 31443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:38.470738 31443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "983ad5b3a2c743e69e4c6ee0bc1eb60a"
format_stamp: "Formatted at 2026-08-09 05:34:38 on dist-test-slave-3kk6"
server_key: "007d59cd273bb06267ab15b0100c969d"
server_key_iv: "0a2997f6d1ff09ac81e955a5c1336c30"
server_key_version: "encryptionkey@0"
I20260809 05:34:38.471632 31443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "983ad5b3a2c743e69e4c6ee0bc1eb60a"
format_stamp: "Formatted at 2026-08-09 05:34:38 on dist-test-slave-3kk6"
server_key: "007d59cd273bb06267ab15b0100c969d"
server_key_iv: "0a2997f6d1ff09ac81e955a5c1336c30"
server_key_version: "encryptionkey@0"
I20260809 05:34:38.477831 31443 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260809 05:34:38.481575 31474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:38.483119 31443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:34:38.483314 31443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "983ad5b3a2c743e69e4c6ee0bc1eb60a"
format_stamp: "Formatted at 2026-08-09 05:34:38 on dist-test-slave-3kk6"
server_key: "007d59cd273bb06267ab15b0100c969d"
server_key_iv: "0a2997f6d1ff09ac81e955a5c1336c30"
server_key_version: "encryptionkey@0"
I20260809 05:34:38.483546 31443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:38.511291 31443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:38.516002 31443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:38.516335 31443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:38.517387 31443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:34:38.518854 31443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:38.518970 31443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:38.519129 31443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:38.519208 31443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:38.556411 31443 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:33383
I20260809 05:34:38.556452 31651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:33383 every 8 connection(s)
I20260809 05:34:38.557927 31443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:34:38.561734 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31443
I20260809 05:34:38.562050 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:34:38.562958 31271 external_mini_cluster.cc:1428] Setting key 2a5773e70d119a484d813f9a3a26bcb7
Aug 09 05:34:38 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253678, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:38.582901 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:38.560929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:33577 (local address 127.30.137.254:35941)
0809 05:34:38.561490 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:38.561498 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:38.562620 (+ 1122us) server_negotiation.cc:407] Connection header received
0809 05:34:38.565114 (+ 2494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:38.565120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:38.565213 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:38.565352 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:38.567059 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:38.567812 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:38.569444 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:38.569860 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:38.570000 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:38.574899 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:38.574910 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:38.574918 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:38.574970 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:38.579000 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:38.579917 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:38.579927 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:38.579934 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:38.580103 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:38.580682 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:38.580688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:38.580693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:38.581139 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:38.581320 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:38.582132 (+ 812us) server_negotiation.cc:299] Negotiation successful
0809 05:34:38.582430 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":255,"threads_started":1}
I20260809 05:34:38.583693 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:38.561459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35941 (local address 127.30.137.193:33577)
0809 05:34:38.562405 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:38.562463 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:38.564762 (+ 2299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:38.565605 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:38.565614 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:38.566167 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:38.566867 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:38.566889 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:38.567992 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:38.567997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:38.569275 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:38.569294 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:38.570116 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:38.570124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:38.570368 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:38.571208 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:38.571246 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:38.574553 (+ 3307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:38.579199 (+ 4646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:38.579208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:38.579223 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:38.579739 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:38.580253 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:38.580258 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:38.580262 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:38.580528 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:38.581291 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:38.581298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:38.581657 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 05:34:38.581912 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 05:34:38.582360 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":365,"threads_started":1}
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:34:38.585638 31654 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35941
I20260809 05:34:38.586025 31654 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:38.586908 31654 heartbeater.cc:511] Master 127.30.137.254:35941 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
I20260809 05:34:38.589613 31351 ts_manager.cc:194] Registered new tserver with Master: 983ad5b3a2c743e69e4c6ee0bc1eb60a (127.30.137.193:33383)
I20260809 05:34:38.591969 31351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:33577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:34:38.614897 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35941
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:38.971726 31666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:38.972054 31666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:38.972160 31666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:38.979987 31666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:38.980136 31666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:38.980260 31666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:34:38.990811 31666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:38.992206 31666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:38.993602 31666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:39.007519 31673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:39.007577 31675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:39.007608 31677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:39.008950 31666 server_base.cc:1034] running on GCE node
I20260809 05:34:39.009860 31666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:39.010910 31666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:39.012204 31666 hybrid_clock.cc:648] HybridClock initialized: now 1786253679012149 us; error 63 us; skew 500 ppm
I20260809 05:34:39.012614 31666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:39.016157 31666 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:34:39.017674 31666 webserver.cc:460] Webserver started at http://127.30.137.194:35767/ using document root <none> and password file <none>
I20260809 05:34:39.018412 31666 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:39.018569 31666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:39.018947 31666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:39.022380 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "e52066115994409095a179dc43fc1da9"
format_stamp: "Formatted at 2026-08-09 05:34:39 on dist-test-slave-3kk6"
server_key: "e310e49bc64dcaae84cc360b9d2c7068"
server_key_iv: "75bbdef534cba5e7531defb32efb0783"
server_key_version: "encryptionkey@0"
I20260809 05:34:39.023288 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "e52066115994409095a179dc43fc1da9"
format_stamp: "Formatted at 2026-08-09 05:34:39 on dist-test-slave-3kk6"
server_key: "e310e49bc64dcaae84cc360b9d2c7068"
server_key_iv: "75bbdef534cba5e7531defb32efb0783"
server_key_version: "encryptionkey@0"
I20260809 05:34:39.028918 31666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:34:39.032826 31691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:39.034175 31666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 05:34:39.034377 31666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "e52066115994409095a179dc43fc1da9"
format_stamp: "Formatted at 2026-08-09 05:34:39 on dist-test-slave-3kk6"
server_key: "e310e49bc64dcaae84cc360b9d2c7068"
server_key_iv: "75bbdef534cba5e7531defb32efb0783"
server_key_version: "encryptionkey@0"
I20260809 05:34:39.034605 31666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:39.058959 31666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:39.063670 31666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:39.064025 31666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:39.065124 31666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:34:39.066643 31666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:39.066785 31666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:39.066964 31666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:39.067049 31666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:39.103235 31666 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:40129
I20260809 05:34:39.103336 31858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:40129 every 8 connection(s)
I20260809 05:34:39.104759 31666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:34:39.106254 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31666
I20260809 05:34:39.106457 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:34:39.106978 31271 external_mini_cluster.cc:1428] Setting key c93aceb1ec67e084aee61c21b7065a42
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
I20260809 05:34:39.128937 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.107835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:58597 (local address 127.30.137.254:35941)
0809 05:34:39.108354 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:39.108359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:39.109419 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 05:34:39.111432 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:39.111437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:39.111521 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:39.111630 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:39.113856 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.114625 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.116585 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.117377 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.117526 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.121415 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:39.121425 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:39.121434 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:39.121485 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:39.125646 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.126425 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.126432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.126436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.126612 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.127097 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.127104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.127110 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.127524 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:39.127686 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:39.128402 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 05:34:39.128655 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":268,"threads_started":1}
I20260809 05:34:39.130008 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.108389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35941 (local address 127.30.137.194:58597)
0809 05:34:39.109257 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:39.109305 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:39.111167 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:39.111969 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:39.111984 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:39.112830 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:39.113670 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:39.113700 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.114866 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.114874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:39.116417 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:39.116431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.117595 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.117600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:39.117805 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.118558 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:39.118581 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:39.121154 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:39.125806 (+ 4652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:39.125814 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:39.125831 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:39.126279 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:39.126769 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:39.126773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:39.126775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:39.126974 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:39.127672 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:39.127679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:39.128005 (+ 326us) client_negotiation.cc:769] Sending connection context
0809 05:34:39.128250 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 05:34:39.128744 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":327,"threads_started":1}
I20260809 05:34:39.131695 31860 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
I20260809 05:34:39.132064 31860 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:39.133111 31860 heartbeater.cc:511] Master 127.30.137.254:35941 requested a full tablet report, sending...
I20260809 05:34:39.135038 31351 ts_manager.cc:194] Registered new tserver with Master: e52066115994409095a179dc43fc1da9 (127.30.137.194:40129)
I20260809 05:34:39.135924 31351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:58597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:34:39.158332 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35941
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:39.511154 31867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:39.511533 31867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:39.511729 31867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:39.519711 31867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:39.519912 31867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:39.520105 31867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:34:39.530486 31867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:39.532049 31867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:39.533668 31867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:39.547451 31880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:39.547806 31884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:39.547683 31882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:39.548838 31867 server_base.cc:1034] running on GCE node
I20260809 05:34:39.549471 31867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:39.550413 31867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:39.551625 31867 hybrid_clock.cc:648] HybridClock initialized: now 1786253679551567 us; error 46 us; skew 500 ppm
I20260809 05:34:39.551945 31867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:39.555281 31867 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:34:39.557369 31867 webserver.cc:460] Webserver started at http://127.30.137.195:40491/ using document root <none> and password file <none>
I20260809 05:34:39.558257 31867 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:39.558422 31867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:39.558841 31867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:39.563042 31867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "d04e1cf234894ea3b74e31b65424b3e4"
format_stamp: "Formatted at 2026-08-09 05:34:39 on dist-test-slave-3kk6"
server_key: "298e9d561dca2923bb34b646259079f9"
server_key_iv: "6d1ac203e3f7ab3a0fc890c91987d514"
server_key_version: "encryptionkey@0"
I20260809 05:34:39.563827 31867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "d04e1cf234894ea3b74e31b65424b3e4"
format_stamp: "Formatted at 2026-08-09 05:34:39 on dist-test-slave-3kk6"
server_key: "298e9d561dca2923bb34b646259079f9"
server_key_iv: "6d1ac203e3f7ab3a0fc890c91987d514"
server_key_version: "encryptionkey@0"
I20260809 05:34:39.569285 31867 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:34:39.572925 31895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:39.574311 31867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:34:39.574460 31867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "d04e1cf234894ea3b74e31b65424b3e4"
format_stamp: "Formatted at 2026-08-09 05:34:39 on dist-test-slave-3kk6"
server_key: "298e9d561dca2923bb34b646259079f9"
server_key_iv: "6d1ac203e3f7ab3a0fc890c91987d514"
server_key_version: "encryptionkey@0"
I20260809 05:34:39.574640 31867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:39.589197 31867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:39.593737 31867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:39.594020 31867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:39.595100 31867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:34:39.595893 31654 heartbeater.cc:503] Master 127.30.137.254:35941 was elected leader, sending a full tablet report...
I20260809 05:34:39.596714 31867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:39.596791 31867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:39.596998 31867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:39.597086 31867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:39.637279 31867 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:35289
I20260809 05:34:39.637327 32067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:35289 every 8 connection(s)
I20260809 05:34:39.638906 31867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:34:39.639338 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31867
I20260809 05:34:39.639595 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:34:39.640262 31271 external_mini_cluster.cc:1428] Setting key 03a4b77c37e00309911e9c6c0fba53d3
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:39.662077 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.642224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:42607 (local address 127.30.137.254:35941)
0809 05:34:39.642789 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:39.642799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:39.644416 (+ 1617us) server_negotiation.cc:407] Connection header received
0809 05:34:39.646361 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:39.646366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:39.646434 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:39.646535 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:39.648358 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.649418 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.651133 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.651565 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.651730 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.655635 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:39.655640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:39.655643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:39.655680 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:39.658732 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.659554 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.659561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.659569 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.659701 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.660198 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.660202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.660205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.660483 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:39.660659 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:39.661412 (+ 753us) server_negotiation.cc:299] Negotiation successful
0809 05:34:39.661651 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":1402,"threads_started":1}
I20260809 05:34:39.663167 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.643346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35941 (local address 127.30.137.195:42607)
0809 05:34:39.644214 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:39.644265 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:39.646094 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:39.646832 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:39.646841 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:39.647414 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:39.648142 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:39.648171 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.649579 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.649584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:39.650984 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:39.650998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.651828 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.651837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:39.652108 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.652884 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:39.652916 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:39.655366 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:39.658903 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:39.658914 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:39.658938 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:39.659415 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:39.659827 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:39.659833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:39.659837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:39.660061 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:39.660624 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:39.660633 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:39.661015 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 05:34:39.661299 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 05:34:39.661897 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":371,"threads_started":1}
I20260809 05:34:39.664603 32068 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35941
I20260809 05:34:39.664902 32068 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:39.665632 32068 heartbeater.cc:511] Master 127.30.137.254:35941 requested a full tablet report, sending...
I20260809 05:34:39.667124 31351 ts_manager.cc:194] Registered new tserver with Master: d04e1cf234894ea3b74e31b65424b3e4 (127.30.137.195:35289)
I20260809 05:34:39.668126 31351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:42607
I20260809 05:34:39.670410 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:39.723526 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.707098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33286 (local address 127.30.137.193:33383)
0809 05:34:39.708057 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:39.708064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:39.708093 (+ 29us) server_negotiation.cc:407] Connection header received
0809 05:34:39.708161 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:39.708168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:39.708397 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:39.708539 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:39.710050 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.711218 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.712652 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.713297 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.713406 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.717612 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:39.717625 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:39.717640 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:39.717717 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:39.720751 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.721413 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.721421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.721423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.721564 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.721938 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.721942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.721944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.722333 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:39.722626 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:39.722969 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 05:34:39.723196 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":603,"threads_started":1}
W20260809 05:34:39.724889 31518 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33286
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:39.776084 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.761638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.30.137.193:33383)
0809 05:34:39.761926 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:39.761933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:39.762201 (+ 268us) server_negotiation.cc:407] Connection header received
0809 05:34:39.762354 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:39.762358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:39.762465 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:39.762565 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:39.764094 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.764825 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.766064 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.766492 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.766593 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.770888 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:39.770893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:39.770897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:39.770934 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:39.773520 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.774195 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.774202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.774206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.774383 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.774786 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.774789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.774792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.775061 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:39.775251 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:39.775516 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 05:34:39.775695 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 05:34:39.777691 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31443
I20260809 05:34:39.791545 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.778760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.30.137.193:33383)
0809 05:34:39.778936 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:39.778940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:39.779327 (+ 387us) server_negotiation.cc:407] Connection header received
0809 05:34:39.779487 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:39.779491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:39.779547 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:39.779658 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:39.780883 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.781590 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.782959 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.783410 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.783509 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.786018 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:39.786023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:39.786026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:39.786060 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:39.788957 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.789705 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.789709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.789712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.789880 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.790245 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.790248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.790250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.790519 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:39.790792 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:39.791085 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 05:34:39.791267 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 05:34:39.792311 31644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:39.880179 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31443
I20260809 05:34:39.905630 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31666
Aug 09 05:34:39 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:34:39.928253 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:39.906628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.30.137.194:40129)
0809 05:34:39.907293 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:39.907301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:39.907327 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:34:39.907391 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:39.907398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:39.907614 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:39.907811 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:39.909122 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.910394 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.912694 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:39.913789 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:39.913912 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:39.921828 (+ 7916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:39.921846 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:39.921862 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:39.921968 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:39.925311 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.926063 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.926070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.926073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.926223 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:39.926626 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:39.926629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:39.926631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:39.927077 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:39.927365 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:39.927583 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 05:34:39.927813 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":313,"threads_started":1}
I20260809 05:34:39.929679 31853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:40.028941 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31666
I20260809 05:34:40.055155 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31867
Aug 09 05:34:40 dist-test-slave-3kk6 krb5kdc[31279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:34:40.072515 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:40.056385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.30.137.195:35289)
0809 05:34:40.057181 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:40.057189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:40.057221 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:34:40.057315 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:40.057322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:40.057542 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:40.057675 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:40.059069 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:40.060154 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:40.061467 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:40.062070 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:40.062171 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:40.066740 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:40.066761 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:40.066784 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:40.066838 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:40.069751 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:40.070444 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:40.070451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:40.070453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:40.070592 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:40.070973 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:40.070977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:40.070979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:40.071343 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:40.071600 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:40.071856 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 05:34:40.072072 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":453,"threads_started":1}
I20260809 05:34:40.073860 32062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:40.165763 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31867
I20260809 05:34:40.193405 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31299
I20260809 05:34:40.194756 31394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:40.303428 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31299
2026-08-09T05:34:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4405 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:34:40 dist-test-slave-3kk6 krb5kdc[32104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:34:40 dist-test-slave-3kk6 krb5kdc[32104](info): set up 2 sockets
Aug 09 05:34:40 dist-test-slave-3kk6 krb5kdc[32104](info): commencing operation
krb5kdc: starting...
W20260809 05:34:41.620569 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.007s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:34:41 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:34:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:34:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:34:41.845005 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39943
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:39943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:42.196924 32121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:42.197355 32121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:42.197532 32121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:42.205248 32121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:34:42.205435 32121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:42.205566 32121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:42.205687 32121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:34:42.205772 32121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:34:42.216416 32121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:39943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:42.217950 32121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:42.219410 32121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:42.231914 32129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:42.231984 32130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:42.231966 32132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:42.233194 32121 server_base.cc:1034] running on GCE node
I20260809 05:34:42.234128 32121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:42.235880 32121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:42.237154 32121 hybrid_clock.cc:648] HybridClock initialized: now 1786253682237112 us; error 75 us; skew 500 ppm
I20260809 05:34:42.237545 32121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:42 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253682, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:42.245430 32121 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:34:42.247524 32121 webserver.cc:460] Webserver started at http://127.30.137.254:41177/ using document root <none> and password file <none>
I20260809 05:34:42.248476 32121 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:42.248646 32121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:42.249013 32121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:42.252499 32121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "faa22bc4ab5d4b0a8e962846fbcfd439"
format_stamp: "Formatted at 2026-08-09 05:34:42 on dist-test-slave-3kk6"
server_key: "43acb143f6a8d64494abd80661e0d354"
server_key_iv: "5b4558e82472b50c35efdcee7a1f1ce2"
server_key_version: "encryptionkey@0"
I20260809 05:34:42.253396 32121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "faa22bc4ab5d4b0a8e962846fbcfd439"
format_stamp: "Formatted at 2026-08-09 05:34:42 on dist-test-slave-3kk6"
server_key: "43acb143f6a8d64494abd80661e0d354"
server_key_iv: "5b4558e82472b50c35efdcee7a1f1ce2"
server_key_version: "encryptionkey@0"
I20260809 05:34:42.259413 32121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:34:42.263473 32141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:42.265053 32121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:34:42.265303 32121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "faa22bc4ab5d4b0a8e962846fbcfd439"
format_stamp: "Formatted at 2026-08-09 05:34:42 on dist-test-slave-3kk6"
server_key: "43acb143f6a8d64494abd80661e0d354"
server_key_iv: "5b4558e82472b50c35efdcee7a1f1ce2"
server_key_version: "encryptionkey@0"
I20260809 05:34:42.265553 32121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:42.280514 32121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:42.285432 32121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:42.285789 32121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:42.306285 32121 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:39943
I20260809 05:34:42.306305 32226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:39943 every 8 connection(s)
I20260809 05:34:42.307857 32121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:34:42.308472 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32121
I20260809 05:34:42.308750 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:34:42.309626 31271 external_mini_cluster.cc:1428] Setting key 69869b69dc82fc6ebe81f22c4bcaf97e
I20260809 05:34:42.312897 32228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:34:42 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:42.327072 32228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439: Bootstrap starting.
I20260809 05:34:42.330936 32228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:42.332098 32228 log.cc:824] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:42.335942 32228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439: No bootstrap required, opened a new log
I20260809 05:34:42.336061 32232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:42.311850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.30.137.254:39943)
0809 05:34:42.312767 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:42.312782 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:42.312838 (+ 56us) server_negotiation.cc:407] Connection header received
0809 05:34:42.315331 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:42.315361 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:42.315871 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:42.316372 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:42.317915 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:42.320125 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:42.321477 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:42.322573 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:42.322835 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:42.327518 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:42.327527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:42.327544 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:42.327585 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:42.330896 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:42.331723 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:42.331738 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:42.331745 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:42.331931 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:42.332383 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:42.332388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:42.332391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:42.333153 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:42.333401 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:42.334147 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 05:34:42.334623 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":420,"threads_started":1}
I20260809 05:34:42.344616 32228 raft_consensus.cc:348] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39943 } }
I20260809 05:34:42.345055 32228 raft_consensus.cc:374] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:42.345139 32228 raft_consensus.cc:729] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa22bc4ab5d4b0a8e962846fbcfd439, State: Initialized, Role: FOLLOWER
I20260809 05:34:42.345726 32228 consensus_queue.cc:260] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39943 } }
I20260809 05:34:42.345952 32228 raft_consensus.cc:388] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:34:42.346053 32228 raft_consensus.cc:482] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:34:42.346190 32228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:42.348647 32228 raft_consensus.cc:504] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39943 } }
I20260809 05:34:42.349309 32228 leader_election.cc:302] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faa22bc4ab5d4b0a8e962846fbcfd439; no voters:
I20260809 05:34:42.349982 32228 leader_election.cc:288] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:34:42.350126 32235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:42.350922 32235 raft_consensus.cc:686] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [term 1 LEADER]: Becoming Leader. State: Replica: faa22bc4ab5d4b0a8e962846fbcfd439, State: Running, Role: LEADER
I20260809 05:34:42.351457 32235 consensus_queue.cc:237] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39943 } }
I20260809 05:34:42.352329 32228 sys_catalog.cc:564] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:34:42.354727 32237 sys_catalog.cc:455] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39943 } } }
I20260809 05:34:42.354843 32238 sys_catalog.cc:455] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faa22bc4ab5d4b0a8e962846fbcfd439. Latest consensus state: current_term: 1 leader_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa22bc4ab5d4b0a8e962846fbcfd439" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39943 } } }
I20260809 05:34:42.355134 32237 sys_catalog.cc:458] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:42.355237 32238 sys_catalog.cc:458] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:42.357023 32245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:34:42.362465 32245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:34:42.373165 32245 catalog_manager.cc:1347] Generated new cluster ID: b923593168a04c2bb40b8067a82cdd99
I20260809 05:34:42.373376 32245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:34:42.391758 32245 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:34:42.393211 32245 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:34:42.409076 32245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faa22bc4ab5d4b0a8e962846fbcfd439: Generated new TSK 0
I20260809 05:34:42.410292 32245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:34:42.467288 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39943
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:42.901998 32269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:42.902361 32269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:42.902540 32269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:42.910571 32269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:42.910785 32269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:42.910974 32269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:34:42.921470 32269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:42.923156 32269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:42.925092 32269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:42.942643 32276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:42.942942 32277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:42.942713 32280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:42.944042 32269 server_base.cc:1034] running on GCE node
I20260809 05:34:42.944936 32269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:42.946334 32269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:42.947605 32269 hybrid_clock.cc:648] HybridClock initialized: now 1786253682947541 us; error 48 us; skew 500 ppm
I20260809 05:34:42.948094 32269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:42 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253682, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:42.952605 32269 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:34:42.954521 32269 webserver.cc:460] Webserver started at http://127.30.137.193:43645/ using document root <none> and password file <none>
I20260809 05:34:42.955302 32269 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:42.955444 32269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:42.955808 32269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:42.959052 32269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "b027305495014ef592a2ea105308b872"
format_stamp: "Formatted at 2026-08-09 05:34:42 on dist-test-slave-3kk6"
server_key: "fc54a842634b1804a3f436163c22c5c0"
server_key_iv: "ad7d25c0d693865d87f3ee9be6a6e005"
server_key_version: "encryptionkey@0"
I20260809 05:34:42.959921 32269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "b027305495014ef592a2ea105308b872"
format_stamp: "Formatted at 2026-08-09 05:34:42 on dist-test-slave-3kk6"
server_key: "fc54a842634b1804a3f436163c22c5c0"
server_key_iv: "ad7d25c0d693865d87f3ee9be6a6e005"
server_key_version: "encryptionkey@0"
I20260809 05:34:42.965478 32269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:34:42.969396 32288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:42.971063 32269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:34:42.971276 32269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "b027305495014ef592a2ea105308b872"
format_stamp: "Formatted at 2026-08-09 05:34:42 on dist-test-slave-3kk6"
server_key: "fc54a842634b1804a3f436163c22c5c0"
server_key_iv: "ad7d25c0d693865d87f3ee9be6a6e005"
server_key_version: "encryptionkey@0"
I20260809 05:34:42.971517 32269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:43.000284 32269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:43.004951 32269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:43.005299 32269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:43.006434 32269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:34:43.008009 32269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:43.008133 32269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:43.008312 32269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:43.008399 32269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:43.046604 32269 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:45485
I20260809 05:34:43.046790 32451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:45485 every 8 connection(s)
I20260809 05:34:43.048076 32269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:34:43.050637 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32269
I20260809 05:34:43.050971 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:34:43.051832 31271 external_mini_cluster.cc:1428] Setting key d67e82684961322e89de1c3c1608efea
Aug 09 05:34:43 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253682, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:43.072655 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:43.051069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:46657 (local address 127.30.137.254:39943)
0809 05:34:43.051674 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:43.051683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:43.052780 (+ 1097us) server_negotiation.cc:407] Connection header received
0809 05:34:43.054857 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:43.054865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:43.054957 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:43.055078 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:43.056697 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.057944 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.059850 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.060593 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.060743 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:43.065847 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:43.065853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:43.065857 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:43.065894 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:43.068952 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:43.069932 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:43.069940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:43.069949 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:43.070088 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:43.070634 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:43.070639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:43.070645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:43.070999 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:43.071206 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:43.071990 (+ 784us) server_negotiation.cc:299] Negotiation successful
0809 05:34:43.072252 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":266,"threads_started":1}
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:34:43.073661 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:43.051689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39943 (local address 127.30.137.193:46657)
0809 05:34:43.052582 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:43.052651 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:43.054493 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:43.055303 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:43.055312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:43.055900 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:43.056499 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:43.056518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.058174 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.058179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:43.059637 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:43.059684 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.060944 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.060953 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:43.061321 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:43.062238 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:43.062277 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:43.065503 (+ 3226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:43.069151 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:43.069161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:43.069181 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:43.069775 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:43.070234 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:43.070239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:43.070243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:43.070492 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:43.071154 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:43.071164 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:43.071562 (+ 398us) client_negotiation.cc:769] Sending connection context
0809 05:34:43.071881 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 05:34:43.072469 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":408,"threads_started":1}
I20260809 05:34:43.075630 32452 heartbeater.cc:347] Connected to a master server at 127.30.137.254:39943
I20260809 05:34:43.076081 32452 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
I20260809 05:34:43.077042 32452 heartbeater.cc:511] Master 127.30.137.254:39943 requested a full tablet report, sending...
I20260809 05:34:43.079682 32169 ts_manager.cc:194] Registered new tserver with Master: b027305495014ef592a2ea105308b872 (127.30.137.193:45485)
I20260809 05:34:43.081911 32169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:46657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:34:43.106202 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39943
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:43.459339 32458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:43.459726 32458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:43.459931 32458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:43.467687 32458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:43.467886 32458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:43.468084 32458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:34:43.479166 32458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:43.480695 32458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:43.482106 32458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:43.496892 32467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:43.497007 32466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:43.496979 32471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:43.498514 32458 server_base.cc:1034] running on GCE node
I20260809 05:34:43.499308 32458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:43.500432 32458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:43.501715 32458 hybrid_clock.cc:648] HybridClock initialized: now 1786253683501667 us; error 46 us; skew 500 ppm
I20260809 05:34:43.502080 32458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:43 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253683, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:43.505769 32458 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:34:43.507714 32458 webserver.cc:460] Webserver started at http://127.30.137.194:46537/ using document root <none> and password file <none>
I20260809 05:34:43.508451 32458 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:43.508616 32458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:43.508972 32458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:43.512198 32458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "27f6f7033e7e43369beb8248df111ed8"
format_stamp: "Formatted at 2026-08-09 05:34:43 on dist-test-slave-3kk6"
server_key: "d9447acd0cd06511ce51d4c33aeecd25"
server_key_iv: "178be8f28ccb08df40715792225daf5c"
server_key_version: "encryptionkey@0"
I20260809 05:34:43.513043 32458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "27f6f7033e7e43369beb8248df111ed8"
format_stamp: "Formatted at 2026-08-09 05:34:43 on dist-test-slave-3kk6"
server_key: "d9447acd0cd06511ce51d4c33aeecd25"
server_key_iv: "178be8f28ccb08df40715792225daf5c"
server_key_version: "encryptionkey@0"
I20260809 05:34:43.518625 32458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:34:43.522475 32481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:43.524040 32458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:34:43.524274 32458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "27f6f7033e7e43369beb8248df111ed8"
format_stamp: "Formatted at 2026-08-09 05:34:43 on dist-test-slave-3kk6"
server_key: "d9447acd0cd06511ce51d4c33aeecd25"
server_key_iv: "178be8f28ccb08df40715792225daf5c"
server_key_version: "encryptionkey@0"
I20260809 05:34:43.524529 32458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:43.536314 32458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:43.541348 32458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:43.541705 32458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:43.542872 32458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:34:43.544359 32458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:43.544482 32458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:43.544642 32458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:43.544734 32458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:43.582556 32458 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:42791
I20260809 05:34:43.582623 32656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:42791 every 8 connection(s)
I20260809 05:34:43.584040 32458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:34:43.589690 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32458
I20260809 05:34:43.589912 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:34:43.590515 31271 external_mini_cluster.cc:1428] Setting key f36e50e726fa4f3be47bfee910c4e70f
Aug 09 05:34:43 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253683, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:43.608929 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:43.587125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:52459 (local address 127.30.137.254:39943)
0809 05:34:43.587665 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:43.587670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:43.588485 (+ 815us) server_negotiation.cc:407] Connection header received
0809 05:34:43.591206 (+ 2721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:43.591217 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:43.591324 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:43.591498 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:43.593540 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.594383 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.597179 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.597718 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.597846 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:43.602339 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:43.602345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:43.602348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:43.602386 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:43.605326 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:43.606228 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:43.606233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:43.606235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:43.606374 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:43.606963 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:43.606971 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:43.606978 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:43.607276 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:43.607454 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:43.608269 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 05:34:43.608528 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":283,"threads_started":1}
I20260809 05:34:43.609994 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:43.587548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39943 (local address 127.30.137.194:52459)
0809 05:34:43.588294 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:43.588334 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:43.590785 (+ 2451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:43.591889 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:43.591898 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:43.592447 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:43.593324 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:43.593354 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.594583 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.594592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:43.596953 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:43.596981 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:43.598001 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:43.598011 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:43.598379 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:43.599315 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:43.599352 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:43.602052 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:43.605526 (+ 3474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:43.605537 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:43.605556 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:43.606055 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:43.606510 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:43.606516 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:43.606520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:43.606825 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:43.607425 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:43.607433 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:43.607852 (+ 419us) client_negotiation.cc:769] Sending connection context
0809 05:34:43.608148 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 05:34:43.608755 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":289,"threads_started":1}
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
I20260809 05:34:43.611795 32659 heartbeater.cc:347] Connected to a master server at 127.30.137.254:39943
I20260809 05:34:43.612102 32659 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:43.612978 32659 heartbeater.cc:511] Master 127.30.137.254:39943 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
I20260809 05:34:43.614774 32169 ts_manager.cc:194] Registered new tserver with Master: 27f6f7033e7e43369beb8248df111ed8 (127.30.137.194:42791)
I20260809 05:34:43.615686 32169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:52459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:34:43.642932 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39943
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:43.999195 32668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:43.999509 32668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:43.999614 32668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:44.007081 32668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:44.007212 32668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:44.007331 32668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:34:44.017371 32668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:44.018688 32668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:44.020076 32668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:44.034353 32681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:44.034442 32677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:44.034437 32678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:44.035869 32668 server_base.cc:1034] running on GCE node
I20260809 05:34:44.036563 32668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:44.037533 32668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:44.038776 32668 hybrid_clock.cc:648] HybridClock initialized: now 1786253684038724 us; error 35 us; skew 500 ppm
I20260809 05:34:44.039130 32668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:44.042846 32668 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:34:44.044608 32668 webserver.cc:460] Webserver started at http://127.30.137.195:41883/ using document root <none> and password file <none>
I20260809 05:34:44.045403 32668 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:44.045575 32668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:44.046034 32668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:44.050526 32668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "78c8a9103c2646a1ad9e2da2a40b8fff"
format_stamp: "Formatted at 2026-08-09 05:34:44 on dist-test-slave-3kk6"
server_key: "980d784c01e907ee5001825426be5957"
server_key_iv: "34a65fb24eddf55fdfb9f27a859fb351"
server_key_version: "encryptionkey@0"
I20260809 05:34:44.051674 32668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "78c8a9103c2646a1ad9e2da2a40b8fff"
format_stamp: "Formatted at 2026-08-09 05:34:44 on dist-test-slave-3kk6"
server_key: "980d784c01e907ee5001825426be5957"
server_key_iv: "34a65fb24eddf55fdfb9f27a859fb351"
server_key_version: "encryptionkey@0"
I20260809 05:34:44.057740 32668 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 05:34:44.061599 32689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:44.063138 32668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:34:44.063335 32668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "78c8a9103c2646a1ad9e2da2a40b8fff"
format_stamp: "Formatted at 2026-08-09 05:34:44 on dist-test-slave-3kk6"
server_key: "980d784c01e907ee5001825426be5957"
server_key_iv: "34a65fb24eddf55fdfb9f27a859fb351"
server_key_version: "encryptionkey@0"
I20260809 05:34:44.063573 32668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:44.076233 32668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:44.080456 32668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:44.080790 32668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:44.081794 32668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:34:44.083303 32668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:44.083377 32668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:44.083511 32668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:44.083566 32668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:44.085304 32452 heartbeater.cc:503] Master 127.30.137.254:39943 was elected leader, sending a full tablet report...
I20260809 05:34:44.124567 32668 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:32791
I20260809 05:34:44.124606 397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:32791 every 8 connection(s)
I20260809 05:34:44.126010 32668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:34:44.127305 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32668
I20260809 05:34:44.127526 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:34:44.128122 31271 external_mini_cluster.cc:1428] Setting key b22752662bc32dc47a2ba87e0c94737d
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:44.147734 403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.129374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:32813 (local address 127.30.137.254:39943)
0809 05:34:44.129905 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.129910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.130817 (+ 907us) server_negotiation.cc:407] Connection header received
0809 05:34:44.132800 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.132805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.132872 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.132964 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.134584 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.135447 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.137078 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.137500 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.137615 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.141307 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.141312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.141315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.141350 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.144629 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.145377 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.145382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.145384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.145515 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.145919 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.145923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.145925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.146209 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.146333 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.147247 (+ 914us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.147433 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":267,"threads_started":1}
I20260809 05:34:44.148728 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.129700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39943 (local address 127.30.137.195:32813)
0809 05:34:44.130630 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:44.130702 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:44.132513 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:44.133193 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:44.133201 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:44.133758 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:44.134403 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.134423 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.135631 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.135635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.136910 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.136929 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.137724 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.137729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.137944 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.138717 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:44.138753 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:44.141101 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:44.144762 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:44.144770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:44.144785 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:44.145248 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:44.145619 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:44.145623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:44.145625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:44.145815 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:44.146391 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:44.146398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:44.146871 (+ 473us) client_negotiation.cc:769] Sending connection context
0809 05:34:44.147140 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 05:34:44.147564 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":407,"threads_started":1}
I20260809 05:34:44.150255 399 heartbeater.cc:347] Connected to a master server at 127.30.137.254:39943
I20260809 05:34:44.150635 399 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:44.151455 399 heartbeater.cc:511] Master 127.30.137.254:39943 requested a full tablet report, sending...
I20260809 05:34:44.153370 32169 ts_manager.cc:194] Registered new tserver with Master: 78c8a9103c2646a1ad9e2da2a40b8fff (127.30.137.195:32791)
I20260809 05:34:44.154177 32169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:32813
I20260809 05:34:44.157007 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:34:44.176887 403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.163344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.30.137.254:39943)
0809 05:34:44.163557 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.163566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.164137 (+ 571us) server_negotiation.cc:407] Connection header received
0809 05:34:44.164282 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.164286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.164366 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.164451 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.166072 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.166983 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.168403 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.168866 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.168962 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.171389 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.171394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.171398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.171431 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.174201 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.174877 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.174881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.174884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.175070 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.175398 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.175401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.175403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.175839 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.176062 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.176323 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.176512 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:34:44.183545 32169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:34:44.190761 32169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:44.230865 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.202927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:32791 (local address 127.0.0.1:39100)
0809 05:34:44.203824 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:44.203873 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:44.204049 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:44.222040 (+ 17991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:44.222049 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:44.222079 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:44.222536 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.222554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.224868 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.224874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.227524 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.227540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.229682 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.229690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.229890 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.229952 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 05:34:44.230111 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 05:34:44.230336 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":63,"thread_start_us":594,"threads_started":1}
I20260809 05:34:44.231158 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.203348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.30.137.195:32791)
0809 05:34:44.221125 (+ 17777us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.221138 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.221189 (+ 51us) server_negotiation.cc:407] Connection header received
0809 05:34:44.221342 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.221356 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.221732 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.222038 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:44.223237 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.224704 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.227963 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.229401 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.229712 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.229898 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.230583 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.230814 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17518,"thread_start_us":17326,"threads_started":1}
I20260809 05:34:44.231451 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.203872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.30.137.193:45485)
0809 05:34:44.223013 (+ 19141us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.223022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.223050 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:34:44.223151 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.223162 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.223441 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.223655 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:44.224301 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.225758 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.228170 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.229709 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.229914 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.230057 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.230959 (+ 902us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.231118 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18935,"thread_start_us":18755,"threads_started":1}
I20260809 05:34:44.231851 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.203685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:42791 (local address 127.0.0.1:51446)
0809 05:34:44.219874 (+ 16189us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:44.219903 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:44.220074 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:44.223044 (+ 2970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:44.223049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:44.223077 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:44.223494 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.223507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.225495 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.225500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.227751 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.227764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.230068 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.230077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.230323 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.230352 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 05:34:44.230439 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 05:34:44.230579 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16006,"mutex_wait_us":2,"thread_start_us":16520,"threads_started":1}
I20260809 05:34:44.232170 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.203028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:45485 (local address 127.0.0.1:57270)
0809 05:34:44.216084 (+ 13056us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:44.216115 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:44.216304 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:44.223692 (+ 7388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:44.223697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:44.223732 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:44.224079 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.224090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.225919 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.225924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.227989 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.228004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.230258 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.230267 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.230540 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.230573 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 05:34:44.230702 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 05:34:44.230842 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12870,"mutex_wait_us":137,"thread_start_us":12724,"threads_started":1}
I20260809 05:34:44.233161 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.204411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.30.137.194:42791)
0809 05:34:44.221550 (+ 17139us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.221560 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.221593 (+ 33us) server_negotiation.cc:407] Connection header received
0809 05:34:44.221706 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.221715 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.222827 (+ 1112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.222995 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:44.223698 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.225319 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.228200 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.229700 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.229887 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.230058 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.231781 (+ 1723us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.231996 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16962,"thread_start_us":16803,"threads_started":1}
I20260809 05:34:44.235651 32355 tablet_service.cc:1467] Processing CreateTablet for tablet 69d908ac939747e893576ae01b52480a (DEFAULT_TABLE table=test-table [id=a0b93acbaee64750b9b1d77415998978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:44.235634 32762 tablet_service.cc:1467] Processing CreateTablet for tablet 69d908ac939747e893576ae01b52480a (DEFAULT_TABLE table=test-table [id=a0b93acbaee64750b9b1d77415998978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:44.236310 32558 tablet_service.cc:1467] Processing CreateTablet for tablet 69d908ac939747e893576ae01b52480a (DEFAULT_TABLE table=test-table [id=a0b93acbaee64750b9b1d77415998978]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:44.237280 32762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69d908ac939747e893576ae01b52480a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:44.237293 32355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69d908ac939747e893576ae01b52480a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:44.237563 32558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69d908ac939747e893576ae01b52480a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:44.249140 424 tablet_bootstrap.cc:492] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: Bootstrap starting.
I20260809 05:34:44.252585 426 tablet_bootstrap.cc:492] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: Bootstrap starting.
I20260809 05:34:44.253078 424 tablet_bootstrap.cc:654] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:44.253628 427 tablet_bootstrap.cc:492] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: Bootstrap starting.
I20260809 05:34:44.254781 424 log.cc:824] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:44.257493 427 tablet_bootstrap.cc:654] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:44.257680 426 tablet_bootstrap.cc:654] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:44.258910 427 log.cc:824] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:44.258920 426 log.cc:824] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:44.259670 424 tablet_bootstrap.cc:492] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: No bootstrap required, opened a new log
I20260809 05:34:44.260022 424 ts_tablet_manager.cc:1397] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260809 05:34:44.262256 427 tablet_bootstrap.cc:492] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: No bootstrap required, opened a new log
I20260809 05:34:44.262534 426 tablet_bootstrap.cc:492] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: No bootstrap required, opened a new log
I20260809 05:34:44.262560 427 ts_tablet_manager.cc:1397] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260809 05:34:44.262858 426 ts_tablet_manager.cc:1397] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260809 05:34:44.271109 427 raft_consensus.cc:348] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.271063 424 raft_consensus.cc:348] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.271523 427 raft_consensus.cc:374] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:44.271539 424 raft_consensus.cc:374] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:44.271618 427 raft_consensus.cc:729] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78c8a9103c2646a1ad9e2da2a40b8fff, State: Initialized, Role: FOLLOWER
I20260809 05:34:44.271631 424 raft_consensus.cc:729] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f6f7033e7e43369beb8248df111ed8, State: Initialized, Role: FOLLOWER
I20260809 05:34:44.272307 427 consensus_queue.cc:260] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.272405 424 consensus_queue.cc:260] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.273533 399 heartbeater.cc:503] Master 127.30.137.254:39943 was elected leader, sending a full tablet report...
I20260809 05:34:44.274101 427 ts_tablet_manager.cc:1428] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 05:34:44.273869 426 raft_consensus.cc:348] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.274308 426 raft_consensus.cc:374] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:44.274420 426 raft_consensus.cc:729] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b027305495014ef592a2ea105308b872, State: Initialized, Role: FOLLOWER
I20260809 05:34:44.275036 426 consensus_queue.cc:260] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.275676 32659 heartbeater.cc:503] Master 127.30.137.254:39943 was elected leader, sending a full tablet report...
I20260809 05:34:44.276242 424 ts_tablet_manager.cc:1428] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: Time spent starting tablet: real 0.016s user 0.009s sys 0.004s
I20260809 05:34:44.278461 426 ts_tablet_manager.cc:1428] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
W20260809 05:34:44.300761 32453 tablet.cc:2369] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:34:44.336469 32660 tablet.cc:2369] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:44.372453 436 raft_consensus.cc:482] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:44.372822 436 raft_consensus.cc:504] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.374617 436 leader_election.cc:288] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78c8a9103c2646a1ad9e2da2a40b8fff (127.30.137.195:32791), 27f6f7033e7e43369beb8248df111ed8 (127.30.137.194:42791)
W20260809 05:34:44.378010 402 tablet.cc:2369] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:44.383047 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.375179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:36341 (local address 127.30.137.194:42791)
0809 05:34:44.375449 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.375457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.376269 (+ 812us) server_negotiation.cc:407] Connection header received
0809 05:34:44.376426 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.376452 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.376540 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.376632 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:44.377543 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.378730 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.380806 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.382082 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.382255 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.382335 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.382598 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.382770 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260809 05:34:44.383378 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.374988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:42791 (local address 127.30.137.193:36341)
0809 05:34:44.376139 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:44.376156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:44.376266 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:44.376678 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:44.376684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:44.376776 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:44.377339 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.377357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.378908 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.378913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.380601 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.380617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.382287 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.382291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.382417 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.382442 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 05:34:44.382530 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 05:34:44.382624 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":218,"spinlock_wait_cycles":22912,"thread_start_us":441,"threads_started":1}
I20260809 05:34:44.383415 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.374988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:32791 (local address 127.30.137.193:45153)
0809 05:34:44.376055 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:44.376077 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:44.376263 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:44.376678 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:44.376684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:44.376731 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:44.377393 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.377403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.378947 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.378951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.380603 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:44.380618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.382540 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.382548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:44.382734 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.382766 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:34:44.382872 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 05:34:44.383002 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":49,"thread_start_us":722,"threads_started":1}
I20260809 05:34:44.383467 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.375122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:45153 (local address 127.30.137.195:32791)
0809 05:34:44.375342 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.375349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.376224 (+ 875us) server_negotiation.cc:407] Connection header received
0809 05:34:44.376426 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.376433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.376515 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.376629 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:44.377543 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.378769 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.380860 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.382091 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.382268 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.382335 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.382923 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.383066 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:34:44.384547 325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d908ac939747e893576ae01b52480a" candidate_uuid: "b027305495014ef592a2ea105308b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" is_pre_election: true
I20260809 05:34:44.384549 32583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d908ac939747e893576ae01b52480a" candidate_uuid: "b027305495014ef592a2ea105308b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27f6f7033e7e43369beb8248df111ed8" is_pre_election: true
I20260809 05:34:44.385175 325 raft_consensus.cc:2454] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b027305495014ef592a2ea105308b872 in term 0.
I20260809 05:34:44.385175 32583 raft_consensus.cc:2454] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b027305495014ef592a2ea105308b872 in term 0.
I20260809 05:34:44.385910 32292 leader_election.cc:302] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27f6f7033e7e43369beb8248df111ed8, b027305495014ef592a2ea105308b872; no voters:
I20260809 05:34:44.386287 436 raft_consensus.cc:2781] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:44.386467 436 raft_consensus.cc:482] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:44.386602 436 raft_consensus.cc:3037] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:44.388916 436 raft_consensus.cc:504] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.389599 436 leader_election.cc:288] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [CANDIDATE]: Term 1 election: Requested vote from peers 78c8a9103c2646a1ad9e2da2a40b8fff (127.30.137.195:32791), 27f6f7033e7e43369beb8248df111ed8 (127.30.137.194:42791)
I20260809 05:34:44.390085 325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d908ac939747e893576ae01b52480a" candidate_uuid: "b027305495014ef592a2ea105308b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff"
I20260809 05:34:44.390128 32583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69d908ac939747e893576ae01b52480a" candidate_uuid: "b027305495014ef592a2ea105308b872" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27f6f7033e7e43369beb8248df111ed8"
I20260809 05:34:44.390336 32583 raft_consensus.cc:3037] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:44.390336 325 raft_consensus.cc:3037] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:44.393811 32583 raft_consensus.cc:2454] T 69d908ac939747e893576ae01b52480a P 27f6f7033e7e43369beb8248df111ed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b027305495014ef592a2ea105308b872 in term 1.
I20260809 05:34:44.393813 325 raft_consensus.cc:2454] T 69d908ac939747e893576ae01b52480a P 78c8a9103c2646a1ad9e2da2a40b8fff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b027305495014ef592a2ea105308b872 in term 1.
I20260809 05:34:44.394299 32289 leader_election.cc:302] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78c8a9103c2646a1ad9e2da2a40b8fff, b027305495014ef592a2ea105308b872; no voters:
I20260809 05:34:44.394641 436 raft_consensus.cc:2781] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:44.395187 436 raft_consensus.cc:686] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [term 1 LEADER]: Becoming Leader. State: Replica: b027305495014ef592a2ea105308b872, State: Running, Role: LEADER
I20260809 05:34:44.395701 436 consensus_queue.cc:237] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } }
I20260809 05:34:44.400657 32164 catalog_manager.cc:5495] T 69d908ac939747e893576ae01b52480a P b027305495014ef592a2ea105308b872 reported cstate change: term changed from 0 to 1, leader changed from <none> to b027305495014ef592a2ea105308b872 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "b027305495014ef592a2ea105308b872" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c8a9103c2646a1ad9e2da2a40b8fff" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 32791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b027305495014ef592a2ea105308b872" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 45485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27f6f7033e7e43369beb8248df111ed8" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42791 } health_report { overall_health: UNKNOWN } } }
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:44.462059 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.446282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.30.137.193:45485)
0809 05:34:44.446552 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.446562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.447159 (+ 597us) server_negotiation.cc:407] Connection header received
0809 05:34:44.447282 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.447286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.447370 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.447469 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.449069 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.449870 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.451171 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.451597 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.451726 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.456141 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.456160 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.456177 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.456223 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.459272 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.459990 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.459997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.459999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.460133 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.460497 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.460500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.460502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.460865 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.461082 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.461500 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.461680 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:44.516957 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.502422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.30.137.193:45485)
0809 05:34:44.502696 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.502703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.502944 (+ 241us) server_negotiation.cc:407] Connection header received
0809 05:34:44.503175 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.503180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.503253 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.503360 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.504836 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.505588 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.506929 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.507557 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.507674 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.511848 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.511854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.511857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.511893 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.514365 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.515087 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.515091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.515093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.515236 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.515622 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.515625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.515627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.515932 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.516159 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.516358 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.516536 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:44.570380 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.555432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.30.137.193:45485)
0809 05:34:44.555700 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.555712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.555872 (+ 160us) server_negotiation.cc:407] Connection header received
0809 05:34:44.556030 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.556034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.556115 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.556206 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.557722 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.558630 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.560071 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.560498 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.560597 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.564915 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.564921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.564924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.564961 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.567794 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.568431 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.568436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.568438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.568566 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.568904 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.568907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.568910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.569234 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.569444 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.569700 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.569973 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 05:34:44.572424 32327 tablet_service.cc:2908] Rejecting scan request for tablet 69d908ac939747e893576ae01b52480a: Uninitialized: clean time has not yet been initialized
I20260809 05:34:44.576925 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32269
I20260809 05:34:44.591257 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.577796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.30.137.193:45485)
0809 05:34:44.577955 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.577959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.578374 (+ 415us) server_negotiation.cc:407] Connection header received
0809 05:34:44.578488 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.578492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.578559 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.578640 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.579869 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.580566 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.581728 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.582329 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.582454 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.584888 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.584895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.584900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.584937 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.588390 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.589152 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.589159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.589163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.589311 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.589745 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.589749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.589751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.590024 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.590303 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.590622 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.590916 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:34:44.592101 32447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:44.685451 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32269
I20260809 05:34:44.710208 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32458
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:34:44.725656 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.711290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.30.137.194:42791)
0809 05:34:44.711489 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.711493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.711515 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:34:44.711574 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.711577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.711639 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.711752 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.712946 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.713769 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.715175 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.715560 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.715655 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.719872 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.719886 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.719902 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.719938 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.722655 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.723498 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.723506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.723508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.723650 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.724049 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.724054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.724057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.724431 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.724803 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.725077 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.725307 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 05:34:44.726402 32646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:44.817031 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32458
I20260809 05:34:44.842325 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32668
Aug 09 05:34:44 dist-test-slave-3kk6 krb5kdc[32104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:34:44.858589 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:44.843277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.30.137.195:32791)
0809 05:34:44.843502 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:44.843508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:44.843528 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:34:44.843689 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:44.843699 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:44.843776 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:44.843897 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:44.845198 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.845950 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.847506 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:44.847953 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:44.848095 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:44.852522 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:44.852538 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:44.852554 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:44.852588 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:44.855706 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.856396 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.856404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.856406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.856544 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:44.856958 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:44.856965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:44.856968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:44.857337 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:44.857645 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:44.857970 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 05:34:44.858206 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 05:34:44.859328 392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:44.949831 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32668
I20260809 05:34:44.975167 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32121
I20260809 05:34:44.976397 32216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:45.087946 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32121
2026-08-09T05:34:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4787 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:34:45 dist-test-slave-3kk6 krb5kdc[479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:34:45 dist-test-slave-3kk6 krb5kdc[479](info): set up 2 sockets
Aug 09 05:34:45 dist-test-slave-3kk6 krb5kdc[479](info): commencing operation
krb5kdc: starting...
W20260809 05:34:46.389853 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:34:46 dist-test-slave-3kk6 krb5kdc[479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:34:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:34:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:34:46.610386 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:45149
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:45149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:46.950762 501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:46.951109 501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:46.951273 501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:46.958832 501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:34:46.958950 501 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 05:34:46.959048 501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:46.959159 501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:46.959257 501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:34:46.959374 501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:34:46.969745 501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:45149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:45149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:46.971166 501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:46.972576 501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:46.984977 508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:46.984923 514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:46.985025 509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:46.986241 501 server_base.cc:1034] running on GCE node
I20260809 05:34:46.987150 501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:46.988526 501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:46.989759 501 hybrid_clock.cc:648] HybridClock initialized: now 1786253686989678 us; error 56 us; skew 500 ppm
I20260809 05:34:46.990054 501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:46 dist-test-slave-3kk6 krb5kdc[479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253686, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:46.993608 501 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:34:46.995368 501 webserver.cc:460] Webserver started at http://127.30.137.254:39311/ using document root <none> and password file <none>
I20260809 05:34:46.996112 501 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:46.996243 501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:46.996610 501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:47.000078 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "e47cc1dca01448b6a6a2cb1216a83c85"
format_stamp: "Formatted at 2026-08-09 05:34:46 on dist-test-slave-3kk6"
server_key: "3c9ee7d0113d66765f82a15e1699a9b2"
server_key_iv: "ee56a4435f4f2f9e0a830c4f7d02a7d1"
server_key_version: "encryptionkey@0"
I20260809 05:34:47.000963 501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "e47cc1dca01448b6a6a2cb1216a83c85"
format_stamp: "Formatted at 2026-08-09 05:34:46 on dist-test-slave-3kk6"
server_key: "3c9ee7d0113d66765f82a15e1699a9b2"
server_key_iv: "ee56a4435f4f2f9e0a830c4f7d02a7d1"
server_key_version: "encryptionkey@0"
I20260809 05:34:47.006511 501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:34:47.010300 523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:47.012009 501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:34:47.012238 501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "e47cc1dca01448b6a6a2cb1216a83c85"
format_stamp: "Formatted at 2026-08-09 05:34:46 on dist-test-slave-3kk6"
server_key: "3c9ee7d0113d66765f82a15e1699a9b2"
server_key_iv: "ee56a4435f4f2f9e0a830c4f7d02a7d1"
server_key_version: "encryptionkey@0"
I20260809 05:34:47.012490 501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:47.038440 501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:47.042886 501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:47.043201 501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:47.066987 501 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:45149
I20260809 05:34:47.067045 613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:45149 every 8 connection(s)
I20260809 05:34:47.068480 501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:34:47.072431 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:47.072569 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 501
I20260809 05:34:47.072839 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:34:47.073516 31271 external_mini_cluster.cc:1428] Setting key 16b4cdfa3b174c5c75a88b743cb38398
I20260809 05:34:47.087611 614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: Bootstrap starting.
Aug 09 05:34:47 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:47.090950 614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:47.092432 614 log.cc:824] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:47.096107 614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: No bootstrap required, opened a new log
I20260809 05:34:47.099967 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:47.075845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.30.137.254:45149)
0809 05:34:47.076717 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:47.076738 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:47.076790 (+ 52us) server_negotiation.cc:407] Connection header received
0809 05:34:47.079500 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:47.079533 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:47.080087 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:47.080593 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:47.082022 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.084121 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.085459 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.086521 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.086736 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:47.091005 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:47.091016 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:47.091038 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:47.091093 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:47.095116 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:47.095836 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:47.095844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:47.095852 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:47.096075 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:47.096518 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:47.096525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:47.096530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:47.097281 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:47.097590 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:47.098026 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 05:34:47.098523 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":387,"threads_started":1}
I20260809 05:34:47.105738 614 raft_consensus.cc:348] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45149 } }
I20260809 05:34:47.106199 614 raft_consensus.cc:374] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:47.106328 614 raft_consensus.cc:729] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47cc1dca01448b6a6a2cb1216a83c85, State: Initialized, Role: FOLLOWER
I20260809 05:34:47.106961 614 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45149 } }
I20260809 05:34:47.107228 614 raft_consensus.cc:388] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:34:47.107383 614 raft_consensus.cc:482] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:34:47.107563 614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:47.109794 614 raft_consensus.cc:504] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45149 } }
I20260809 05:34:47.110316 614 leader_election.cc:302] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e47cc1dca01448b6a6a2cb1216a83c85; no voters:
I20260809 05:34:47.110960 614 leader_election.cc:288] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:34:47.111061 621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:47.111908 621 raft_consensus.cc:686] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [term 1 LEADER]: Becoming Leader. State: Replica: e47cc1dca01448b6a6a2cb1216a83c85, State: Running, Role: LEADER
I20260809 05:34:47.112476 621 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45149 } }
I20260809 05:34:47.113168 614 sys_catalog.cc:564] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:34:47.115490 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47cc1dca01448b6a6a2cb1216a83c85. Latest consensus state: current_term: 1 leader_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45149 } } }
I20260809 05:34:47.115993 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:47.116693 623 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47cc1dca01448b6a6a2cb1216a83c85" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45149 } } }
I20260809 05:34:47.118417 623 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:47.120059 632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:34:47.125487 632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:34:47.134074 632 catalog_manager.cc:1347] Generated new cluster ID: 1bbcee0c319447dd9584480315cb1df0
I20260809 05:34:47.134198 632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:34:47.157769 632 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:34:47.159760 632 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:34:47.177127 632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: Generated new TSK 0
I20260809 05:34:47.178272 632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:34:47.243026 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45149
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:47.591959 650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:47.592321 650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:47.592504 650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:47.600037 650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:47.600217 650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:47.600364 650 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:34:47.600505 650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:34:47.610852 650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:47.612212 650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:47.613783 650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:47.628062 663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:47.628125 660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:47.628438 658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:47.629446 650 server_base.cc:1034] running on GCE node
I20260809 05:34:47.630072 650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:47.631098 650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:47.632366 650 hybrid_clock.cc:648] HybridClock initialized: now 1786253687632303 us; error 35 us; skew 500 ppm
I20260809 05:34:47.632681 650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:47 dist-test-slave-3kk6 krb5kdc[479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253687, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:47.639663 650 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:34:47.641471 650 webserver.cc:460] Webserver started at http://127.30.137.193:37569/ using document root <none> and password file <none>
I20260809 05:34:47.642239 650 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:47.642392 650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:47.642822 650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:47.645902 650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "3e27a4542ada44b1a280020562851455"
format_stamp: "Formatted at 2026-08-09 05:34:47 on dist-test-slave-3kk6"
server_key: "9175b4fc2e4a4df7bef7703369116a36"
server_key_iv: "9a6729f87af2f98bd06658d1b7c6e483"
server_key_version: "encryptionkey@0"
I20260809 05:34:47.646864 650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "3e27a4542ada44b1a280020562851455"
format_stamp: "Formatted at 2026-08-09 05:34:47 on dist-test-slave-3kk6"
server_key: "9175b4fc2e4a4df7bef7703369116a36"
server_key_iv: "9a6729f87af2f98bd06658d1b7c6e483"
server_key_version: "encryptionkey@0"
I20260809 05:34:47.652303 650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 05:34:47.656033 675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:47.657403 650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:34:47.657610 650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "3e27a4542ada44b1a280020562851455"
format_stamp: "Formatted at 2026-08-09 05:34:47 on dist-test-slave-3kk6"
server_key: "9175b4fc2e4a4df7bef7703369116a36"
server_key_iv: "9a6729f87af2f98bd06658d1b7c6e483"
server_key_version: "encryptionkey@0"
I20260809 05:34:47.657904 650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:47.694299 650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:47.699349 650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:47.699666 650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:47.703239 650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:47.703374 650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:47.703545 650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:47.703648 650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:34:47 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253687, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:47.739138 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:47.704139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:46389 (local address 127.30.137.254:45149)
0809 05:34:47.707748 (+ 3609us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:47.707768 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:47.709340 (+ 1572us) server_negotiation.cc:407] Connection header received
0809 05:34:47.712100 (+ 2760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:47.712110 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:47.712230 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:47.713348 (+ 1118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:47.715469 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.717009 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.720802 (+ 3793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.721770 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.721910 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:47.728942 (+ 7032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:47.728953 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:47.728962 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:47.729019 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:47.733922 (+ 4903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:47.735378 (+ 1456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:47.735387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:47.735394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:47.735605 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:47.736286 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:47.736295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:47.736300 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:47.736778 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:47.736974 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:47.738149 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0809 05:34:47.738505 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3453,"thread_start_us":3321,"threads_started":1}
I20260809 05:34:47.740705 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:47.704918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.30.137.193:46389)
0809 05:34:47.708543 (+ 3625us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:47.708605 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:47.711602 (+ 2997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:47.712801 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:47.712817 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:47.713950 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:47.715138 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:47.715185 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.717246 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.717255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:47.720563 (+ 3308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:47.720588 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.722450 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.722460 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:47.722898 (+ 438us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:47.724041 (+ 1143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:47.724071 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:47.728542 (+ 4471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:47.734194 (+ 5652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:47.734209 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:47.734240 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:47.735170 (+ 930us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:47.735771 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:47.735779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:47.735785 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:47.736117 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:47.736949 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:47.736961 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:47.737603 (+ 642us) client_negotiation.cc:769] Sending connection context
0809 05:34:47.737991 (+ 388us) client_negotiation.cc:240] Negotiation successful
0809 05:34:47.738763 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3312,"thread_start_us":3072,"threads_started":1}
I20260809 05:34:47.772264 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:47.750765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.30.137.193:55937)
0809 05:34:47.751731 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:47.751758 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:47.751953 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:47.752435 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:47.752442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:47.753278 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:47.753770 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:47.753788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.755776 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.755785 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:47.758332 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:47.758357 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.759853 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.759863 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:47.760166 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:47.761163 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:47.761197 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:47.762948 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:47.768229 (+ 5281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:47.768238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:47.768246 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:47.768992 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:47.769540 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:47.769548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:47.769555 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:47.769759 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:47.770523 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:47.770531 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:47.770850 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 05:34:47.771079 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 05:34:47.771588 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":566,"threads_started":1}
I20260809 05:34:47.773722 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:47.750737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:55937 (local address 127.30.137.254:45149)
0809 05:34:47.750970 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:47.750979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:47.751893 (+ 914us) server_negotiation.cc:407] Connection header received
0809 05:34:47.752113 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:47.752120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:47.752234 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:47.752383 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:47.753976 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.755525 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.758558 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:47.759587 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:47.759758 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:47.763343 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:47.763354 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:47.763363 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:47.763420 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:47.768052 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:47.769138 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:47.769148 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:47.769154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:47.769378 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:47.769895 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:47.769903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:47.769910 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:47.770356 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:47.770555 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:47.772849 (+ 2294us) server_negotiation.cc:299] Negotiation successful
0809 05:34:47.773159 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 05:34:47.793752 650 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:34191
I20260809 05:34:47.793778 855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:34191 every 8 connection(s)
I20260809 05:34:47.795293 650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:34:47.797870 857 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45149
I20260809 05:34:47.798182 857 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:47.799238 857 heartbeater.cc:511] Master 127.30.137.254:45149 requested a full tablet report, sending...
I20260809 05:34:47.799698 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 650
I20260809 05:34:47.799937 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:34:47.800727 31271 external_mini_cluster.cc:1428] Setting key bb5f9ed6046067dd94dd5a19433b401c
I20260809 05:34:47.802322 549 ts_manager.cc:194] Registered new tserver with Master: 3e27a4542ada44b1a280020562851455 (127.30.137.193:34191)
I20260809 05:34:47.805109 549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:46389
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:34:47.850937 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45149
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:48.200158 862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:48.200460 862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:48.200600 862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:48.208164 862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:48.208305 862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:48.208429 862 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:34:48.208582 862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:34:48.219660 862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:48.220997 862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:48.222364 862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:48.236284 868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:48.236399 877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:48.236438 872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:48.237409 862 server_base.cc:1034] running on GCE node
I20260809 05:34:48.238055 862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:48.239013 862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:48.240238 862 hybrid_clock.cc:648] HybridClock initialized: now 1786253688240185 us; error 30 us; skew 500 ppm
I20260809 05:34:48.240504 862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:48 dist-test-slave-3kk6 krb5kdc[479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:48.244024 862 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:34:48.245509 862 webserver.cc:460] Webserver started at http://127.30.137.194:37211/ using document root <none> and password file <none>
I20260809 05:34:48.246217 862 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:48.246367 862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:48.246781 862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:48.249707 862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "86a082d448e1458ca1fd577b51893f0e"
format_stamp: "Formatted at 2026-08-09 05:34:48 on dist-test-slave-3kk6"
server_key: "a4afc7a297c1e7368d492bf93f540e9d"
server_key_iv: "bb99174581628999bd571c8dd5986968"
server_key_version: "encryptionkey@0"
I20260809 05:34:48.250540 862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "86a082d448e1458ca1fd577b51893f0e"
format_stamp: "Formatted at 2026-08-09 05:34:48 on dist-test-slave-3kk6"
server_key: "a4afc7a297c1e7368d492bf93f540e9d"
server_key_iv: "bb99174581628999bd571c8dd5986968"
server_key_version: "encryptionkey@0"
I20260809 05:34:48.255733 862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:34:48.259259 884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:48.260519 862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:34:48.260703 862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "86a082d448e1458ca1fd577b51893f0e"
format_stamp: "Formatted at 2026-08-09 05:34:48 on dist-test-slave-3kk6"
server_key: "a4afc7a297c1e7368d492bf93f540e9d"
server_key_iv: "bb99174581628999bd571c8dd5986968"
server_key_version: "encryptionkey@0"
I20260809 05:34:48.260957 862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:48.278313 862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:48.282987 862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:48.283279 862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:48.285974 862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:48.286114 862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:48.286257 862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:48.286345 862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:34:48 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:48.310160 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.286771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:54091 (local address 127.30.137.254:45149)
0809 05:34:48.287414 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.287419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.288501 (+ 1082us) server_negotiation.cc:407] Connection header received
0809 05:34:48.290618 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.290624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.290768 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.290905 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:48.293367 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.294579 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.297247 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.297932 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.298097 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.302889 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:48.302895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:48.302899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:48.302933 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:48.306535 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.307616 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.307624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.307628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.307769 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.308287 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.308292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.308295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.308587 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:48.308768 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:48.309526 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 05:34:48.309751 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":418,"threads_started":1}
I20260809 05:34:48.311590 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.287430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.30.137.194:54091)
0809 05:34:48.288315 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.288360 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.290312 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.291168 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.291180 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.292074 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:48.293076 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.293112 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.294803 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.294809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.297063 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.297088 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.298204 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.298211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.298470 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.299446 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:48.299469 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:48.302595 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:48.306712 (+ 4117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.306721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.306738 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.307471 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.307912 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.307917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.307921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.308135 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.308731 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:48.308740 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:48.309117 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 05:34:48.309427 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 05:34:48.309996 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":403,"threads_started":1}
I20260809 05:34:48.341264 862 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:37385
I20260809 05:34:48.341365 1066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:37385 every 8 connection(s)
I20260809 05:34:48.343106 862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:34:48.343654 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 862
I20260809 05:34:48.343919 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:34:48.344233 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.318790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:54335 (local address 127.30.137.254:45149)
0809 05:34:48.318989 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.319000 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.327734 (+ 8734us) server_negotiation.cc:407] Connection header received
0809 05:34:48.327986 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.327995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.328112 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.328235 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:48.329991 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.331281 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.333013 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.333787 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.333957 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.336599 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:48.336606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:48.336611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:48.336659 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:48.340868 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.341839 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.341845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.341849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.342005 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.342386 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.342391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.342394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.342917 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:48.343110 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:48.343586 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 05:34:48.343833 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:34:48.344326 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.319111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.30.137.194:54335)
0809 05:34:48.327561 (+ 8450us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.327585 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.327805 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.328322 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.328329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.329144 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:48.329691 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.329711 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.331436 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.331440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.332794 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.332810 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.334035 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.334044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.334282 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.335173 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:48.335203 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:48.336347 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:48.341062 (+ 4715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.341073 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.341079 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.341708 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.342128 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.342132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.342134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.342264 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.343069 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:48.343073 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:48.343294 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 05:34:48.343470 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 05:34:48.343847 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8281,"thread_start_us":8143,"threads_started":1}
I20260809 05:34:48.344619 31271 external_mini_cluster.cc:1428] Setting key 8e85ed88bdebcd1ca76301d3157e24b7
I20260809 05:34:48.351395 1068 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45149
I20260809 05:34:48.351933 1068 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:48.352630 1068 heartbeater.cc:511] Master 127.30.137.254:45149 requested a full tablet report, sending...
I20260809 05:34:48.354056 549 ts_manager.cc:194] Registered new tserver with Master: 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385)
I20260809 05:34:48.354995 549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:54091
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:34:48.397821 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45149
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:48.739850 1077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:48.740226 1077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:48.740407 1077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:48.747855 1077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:48.748027 1077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:48.748200 1077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:34:48.748349 1077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:34:48.758177 1077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:48.759593 1077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:48.761031 1077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:48.775643 1086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:48.775873 1084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:48.775822 1083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:48.776822 1077 server_base.cc:1034] running on GCE node
I20260809 05:34:48.777474 1077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:48.778436 1077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:48.779726 1077 hybrid_clock.cc:648] HybridClock initialized: now 1786253688779628 us; error 48 us; skew 500 ppm
I20260809 05:34:48.780055 1077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:48 dist-test-slave-3kk6 krb5kdc[479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:48.783574 1077 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:34:48.785226 1077 webserver.cc:460] Webserver started at http://127.30.137.195:42101/ using document root <none> and password file <none>
I20260809 05:34:48.786000 1077 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:48.786142 1077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:48.786492 1077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:48.789600 1077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "66c883e354e74a418c6bc6db44c07173"
format_stamp: "Formatted at 2026-08-09 05:34:48 on dist-test-slave-3kk6"
server_key: "2ef0a88ad8a0646efa6157f8aae0fd40"
server_key_iv: "8e23945d44625b07302d9ab0d6759538"
server_key_version: "encryptionkey@0"
I20260809 05:34:48.790493 1077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "66c883e354e74a418c6bc6db44c07173"
format_stamp: "Formatted at 2026-08-09 05:34:48 on dist-test-slave-3kk6"
server_key: "2ef0a88ad8a0646efa6157f8aae0fd40"
server_key_iv: "8e23945d44625b07302d9ab0d6759538"
server_key_version: "encryptionkey@0"
I20260809 05:34:48.796303 1077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 05:34:48.800143 1095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:48.801542 1077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:34:48.801751 1077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "66c883e354e74a418c6bc6db44c07173"
format_stamp: "Formatted at 2026-08-09 05:34:48 on dist-test-slave-3kk6"
server_key: "2ef0a88ad8a0646efa6157f8aae0fd40"
server_key_iv: "8e23945d44625b07302d9ab0d6759538"
server_key_version: "encryptionkey@0"
I20260809 05:34:48.802008 1077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:48.809451 857 heartbeater.cc:503] Master 127.30.137.254:45149 was elected leader, sending a full tablet report...
I20260809 05:34:48.833429 1077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:48.837702 1077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:48.837957 1077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:48.840926 1077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:48.841020 1077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:48.841185 1077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:48.841255 1077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:34:48 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:48.866533 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.842809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:49559 (local address 127.30.137.254:45149)
0809 05:34:48.843005 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.843012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.843645 (+ 633us) server_negotiation.cc:407] Connection header received
0809 05:34:48.846020 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.846029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.846148 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.846297 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:48.848825 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.849994 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.852851 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.853602 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.853771 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.858958 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:48.858964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:48.858968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:48.859007 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:48.862923 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.863724 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.863730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.863734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.863934 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.864508 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.864513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.864516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.864813 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:48.864984 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:48.865854 (+ 870us) server_negotiation.cc:299] Negotiation successful
0809 05:34:48.866085 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 05:34:48.867607 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.842146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.30.137.195:49559)
0809 05:34:48.843457 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.843511 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.845697 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.846584 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.846595 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.847482 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:48.848510 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.848544 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.850199 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.850205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.852657 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.852678 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.853907 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.853917 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.854246 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.855253 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:48.855287 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:48.858612 (+ 3325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:48.863078 (+ 4466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.863088 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.863111 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.863568 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.864114 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.864118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.864120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.864378 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.864963 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:48.864971 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:48.865466 (+ 495us) client_negotiation.cc:769] Sending connection context
0809 05:34:48.865759 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 05:34:48.866323 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"thread_start_us":545,"threads_started":1}
I20260809 05:34:48.890269 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.873980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:42235 (local address 127.30.137.254:45149)
0809 05:34:48.874155 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.874159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.874963 (+ 804us) server_negotiation.cc:407] Connection header received
0809 05:34:48.875159 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.875168 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.875233 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.875343 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:48.876316 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.877088 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.879038 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.879672 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.879823 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.882531 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:48.882540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:48.882548 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:48.882599 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:48.887133 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.887892 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.887902 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.887908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.888129 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.888669 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.888676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.888678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.889014 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:48.889182 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:48.889652 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 05:34:48.889890 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 05:34:48.890379 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.874130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.30.137.195:42235)
0809 05:34:48.874848 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.874867 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.875032 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.875381 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.875388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.875851 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:48.876163 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.876172 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.877238 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.877247 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.878893 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.878915 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.879910 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.879917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.880146 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.881087 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:48.881110 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:48.882257 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:48.887299 (+ 5042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.887307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.887312 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.887749 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.888291 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:48.888300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:48.888305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:48.888535 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:48.889146 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:48.889151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:48.889362 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 05:34:48.889537 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 05:34:48.889923 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":379,"threads_started":1}
I20260809 05:34:48.901846 1077 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:44683
I20260809 05:34:48.901882 1288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:44683 every 8 connection(s)
I20260809 05:34:48.903218 1077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:34:48.905405 1289 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45149
I20260809 05:34:48.905679 1289 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:48.906351 1289 heartbeater.cc:511] Master 127.30.137.254:45149 requested a full tablet report, sending...
I20260809 05:34:48.907816 549 ts_manager.cc:194] Registered new tserver with Master: 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
I20260809 05:34:48.908617 549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:49559
I20260809 05:34:48.909480 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 1077
I20260809 05:34:48.909687 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:34:48.910261 31271 external_mini_cluster.cc:1428] Setting key 04da82a0f28a4e44d04b7dd280cad76a
I20260809 05:34:48.913241 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 05:34:48 dist-test-slave-3kk6 krb5kdc[479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 05:34:48 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:48.965582 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.951242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.30.137.254:45149)
0809 05:34:48.951465 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.951470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.951825 (+ 355us) server_negotiation.cc:407] Connection header received
0809 05:34:48.951990 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.951994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.952060 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.952156 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:48.953605 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.954353 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.955688 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.956084 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.956194 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:48.960321 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:48.960327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:48.960330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:48.960365 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:48.963129 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.963704 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.963708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.963710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.963828 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:48.964272 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:48.964275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:48.964278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:48.964612 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:48.964844 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:48.965083 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 05:34:48.965249 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:34:48.970255 549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:34:48.977242 549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:49.001772 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.989450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:44683 (local address 127.0.0.1:35898)
0809 05:34:48.990451 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.990507 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.990752 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.993638 (+ 2886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.993646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.993680 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:48.994081 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.994095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.996538 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.996543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.998932 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.998945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.000692 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.000699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.000905 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.000957 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.001108 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.001306 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":37,"thread_start_us":495,"threads_started":1}
I20260809 05:34:49.002498 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.989450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37385 (local address 127.0.0.1:53244)
0809 05:34:48.991843 (+ 2393us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.991865 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.992014 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.994579 (+ 2565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.994587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.994621 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:48.995169 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.995182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.997081 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.997086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:48.999541 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.999556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.001620 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.001628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.001794 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.001836 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.001918 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.002055 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2178,"mutex_wait_us":215,"thread_start_us":1790,"threads_started":1}
I20260809 05:34:49.002697 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.989516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.30.137.194:37385)
0809 05:34:48.993825 (+ 4309us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.993836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.993871 (+ 35us) server_negotiation.cc:407] Connection header received
0809 05:34:48.994001 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.994013 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.994295 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.995326 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:48.995520 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.996922 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.999731 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.001119 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.001297 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.001462 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.002198 (+ 736us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.002377 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4098,"thread_start_us":3925,"threads_started":1}
I20260809 05:34:49.003095 1321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.992009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:34191 (local address 127.0.0.1:48266)
0809 05:34:48.993012 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:48.993035 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:48.993185 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:48.995878 (+ 2693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:48.995882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:48.995908 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:48.996227 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:48.996236 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.997923 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.998242 (+ 319us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.000205 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.000216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.002250 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.002258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.002436 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.002473 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.002546 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.002698 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":97,"thread_start_us":777,"threads_started":1}
I20260809 05:34:49.003136 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.990042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.30.137.195:44683)
0809 05:34:48.992795 (+ 2753us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.992807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.992852 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:34:48.993006 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.993018 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.993345 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.994295 (+ 950us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:48.994708 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.996392 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:48.999104 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.000472 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.000692 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.000884 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.001191 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.001363 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2527,"thread_start_us":2294,"threads_started":1}
I20260809 05:34:49.004173 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:48.992582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.30.137.193:34191)
0809 05:34:48.995301 (+ 2719us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:48.995309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:48.995339 (+ 30us) server_negotiation.cc:407] Connection header received
0809 05:34:48.995439 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:48.995448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:48.995649 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:48.995863 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:48.996422 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:48.997743 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.000376 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.001816 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.003481 (+ 1665us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.003632 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.003724 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.003876 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2552,"thread_start_us":2406,"threads_started":1}
I20260809 05:34:49.006819 959 tablet_service.cc:1467] Processing CreateTablet for tablet 481400dadb3e45bd884741f41b7ec9de (DEFAULT_TABLE table=test-table [id=86637a575c034aaca43a285a3d031628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:49.006824 1175 tablet_service.cc:1467] Processing CreateTablet for tablet 481400dadb3e45bd884741f41b7ec9de (DEFAULT_TABLE table=test-table [id=86637a575c034aaca43a285a3d031628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:49.007529 756 tablet_service.cc:1467] Processing CreateTablet for tablet 481400dadb3e45bd884741f41b7ec9de (DEFAULT_TABLE table=test-table [id=86637a575c034aaca43a285a3d031628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:49.008337 1175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481400dadb3e45bd884741f41b7ec9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.008355 959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481400dadb3e45bd884741f41b7ec9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.008786 756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481400dadb3e45bd884741f41b7ec9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.020453 1325 tablet_bootstrap.cc:492] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Bootstrap starting.
I20260809 05:34:49.023468 1327 tablet_bootstrap.cc:492] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Bootstrap starting.
I20260809 05:34:49.025710 1325 tablet_bootstrap.cc:654] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.027199 1325 log.cc:824] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:49.027256 1327 tablet_bootstrap.cc:654] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.027432 1326 tablet_bootstrap.cc:492] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Bootstrap starting.
I20260809 05:34:49.028595 1327 log.cc:824] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:49.030912 1326 tablet_bootstrap.cc:654] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.031525 1325 tablet_bootstrap.cc:492] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: No bootstrap required, opened a new log
I20260809 05:34:49.031816 1325 ts_tablet_manager.cc:1397] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260809 05:34:49.032588 1326 log.cc:824] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:49.034072 1327 tablet_bootstrap.cc:492] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: No bootstrap required, opened a new log
I20260809 05:34:49.034351 1327 ts_tablet_manager.cc:1397] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 05:34:49.035215 1326 tablet_bootstrap.cc:492] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: No bootstrap required, opened a new log
I20260809 05:34:49.035488 1326 ts_tablet_manager.cc:1397] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 05:34:49.042778 1325 raft_consensus.cc:348] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.043325 1325 raft_consensus.cc:374] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.043089 1326 raft_consensus.cc:348] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.043500 1325 raft_consensus.cc:729] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c883e354e74a418c6bc6db44c07173, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.043519 1326 raft_consensus.cc:374] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.043686 1326 raft_consensus.cc:729] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a082d448e1458ca1fd577b51893f0e, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.044342 1326 consensus_queue.cc:260] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.044323 1325 consensus_queue.cc:260] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.045161 1327 raft_consensus.cc:348] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.045657 1327 raft_consensus.cc:374] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.045696 1289 heartbeater.cc:503] Master 127.30.137.254:45149 was elected leader, sending a full tablet report...
I20260809 05:34:49.045828 1327 raft_consensus.cc:729] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e27a4542ada44b1a280020562851455, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.046201 1068 heartbeater.cc:503] Master 127.30.137.254:45149 was elected leader, sending a full tablet report...
I20260809 05:34:49.046908 1326 ts_tablet_manager.cc:1428] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 05:34:49.048029 1325 ts_tablet_manager.cc:1428] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260809 05:34:49.048012 1327 consensus_queue.cc:260] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.049896 1327 ts_tablet_manager.cc:1428] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260809 05:34:49.102450 1070 tablet.cc:2369] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:49.152397 1333 raft_consensus.cc:482] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:49.152777 1333 raft_consensus.cc:504] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.154623 1333 leader_election.cc:288] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385), 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
W20260809 05:34:49.155033 1290 tablet.cc:2369] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:49.162128 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.156361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:56695 (local address 127.30.137.194:37385)
0809 05:34:49.156566 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.156570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.156587 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:34:49.156653 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.156657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.156715 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.156794 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:49.157485 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.158252 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.160249 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.161069 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.161273 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.161381 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.161720 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.161859 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 05:34:49.162277 1337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.154944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37385 (local address 127.30.137.193:56695)
0809 05:34:49.156016 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.156040 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.156224 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.156852 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.156857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.156882 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:49.157300 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.157311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.158412 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.158417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.160081 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.160094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.161362 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.161369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.161525 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.161551 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.161692 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.161831 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":3,"thread_start_us":1119,"threads_started":1}
I20260809 05:34:49.162787 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.155164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:59991 (local address 127.30.137.195:44683)
0809 05:34:49.155416 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.155427 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.156485 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 05:34:49.156680 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.156684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.156743 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.156865 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:49.157583 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.158286 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.160259 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.161602 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.161781 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.161832 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.162367 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.162506 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:34:49.163410 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.155099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:44683 (local address 127.30.137.193:59991)
0809 05:34:49.156364 (+ 1265us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.156384 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.156530 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.156912 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.156917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.156949 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:49.157408 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.157424 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.158462 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.158466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.160084 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.160097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.162015 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.162023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.162153 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.162175 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.163023 (+ 848us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.163141 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"mutex_wait_us":22,"thread_start_us":530,"threads_started":1}
I20260809 05:34:49.163635 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481400dadb3e45bd884741f41b7ec9de" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a082d448e1458ca1fd577b51893f0e" is_pre_election: true
I20260809 05:34:49.164201 994 raft_consensus.cc:2454] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 0.
I20260809 05:34:49.164343 1213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481400dadb3e45bd884741f41b7ec9de" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c883e354e74a418c6bc6db44c07173" is_pre_election: true
I20260809 05:34:49.164980 676 leader_election.cc:302] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e27a4542ada44b1a280020562851455, 86a082d448e1458ca1fd577b51893f0e; no voters:
I20260809 05:34:49.164983 1213 raft_consensus.cc:2454] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 0.
I20260809 05:34:49.165372 1333 raft_consensus.cc:2781] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:49.165561 1333 raft_consensus.cc:482] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:49.165689 1333 raft_consensus.cc:3037] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.168183 1333 raft_consensus.cc:504] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.168782 1333 leader_election.cc:288] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 election: Requested vote from peers 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385), 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
I20260809 05:34:49.169269 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481400dadb3e45bd884741f41b7ec9de" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a082d448e1458ca1fd577b51893f0e"
I20260809 05:34:49.169298 1213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481400dadb3e45bd884741f41b7ec9de" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c883e354e74a418c6bc6db44c07173"
I20260809 05:34:49.169493 994 raft_consensus.cc:3037] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.169482 1213 raft_consensus.cc:3037] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.172011 994 raft_consensus.cc:2454] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 1.
I20260809 05:34:49.172021 1213 raft_consensus.cc:2454] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 1.
I20260809 05:34:49.172436 676 leader_election.cc:302] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e27a4542ada44b1a280020562851455, 86a082d448e1458ca1fd577b51893f0e; no voters:
I20260809 05:34:49.172745 1333 raft_consensus.cc:2781] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:49.173228 1333 raft_consensus.cc:686] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 1 LEADER]: Becoming Leader. State: Replica: 3e27a4542ada44b1a280020562851455, State: Running, Role: LEADER
I20260809 05:34:49.173729 1333 consensus_queue.cc:237] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } }
I20260809 05:34:49.178728 549 catalog_manager.cc:5495] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e27a4542ada44b1a280020562851455 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } health_report { overall_health: HEALTHY } } }
I20260809 05:34:49.200138 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.195080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.30.137.193:34191)
0809 05:34:49.195269 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.195273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.195294 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:34:49.195589 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.195593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.195690 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.195820 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:49.196162 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.196863 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.198156 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.198553 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.198642 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.199162 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.199640 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.199757 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.199868 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.199935 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 05:34:49.208597 994 raft_consensus.cc:1261] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Refusing update from remote peer 3e27a4542ada44b1a280020562851455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:49.208602 1213 raft_consensus.cc:1261] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Refusing update from remote peer 3e27a4542ada44b1a280020562851455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:49.209896 1333 consensus_queue.cc:1035] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:49.210280 1341 consensus_queue.cc:1035] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:49.224035 1346 mvcc.cc:205] Tried to move back new op lower bound from 7316495110987075584 to 7316495110859390976. Current Snapshot: MvccSnapshot[applied={T|T < 7316495110987075584}]
I20260809 05:34:49.245826 547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:49.246130 547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:49.250417 547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: Sending DeleteTablet for 3 replicas of tablet 481400dadb3e45bd884741f41b7ec9de
I20260809 05:34:49.251837 1175 tablet_service.cc:1514] Processing DeleteTablet for tablet 481400dadb3e45bd884741f41b7ec9de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:49 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:35898
I20260809 05:34:49.251840 959 tablet_service.cc:1514] Processing DeleteTablet for tablet 481400dadb3e45bd884741f41b7ec9de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:49 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:53244
I20260809 05:34:49.252198 756 tablet_service.cc:1514] Processing DeleteTablet for tablet 481400dadb3e45bd884741f41b7ec9de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:49 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:48266
I20260809 05:34:49.254336 547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 05:34:49.254899 1363 tablet_replica.cc:330] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: stopping tablet replica
I20260809 05:34:49.254905 1361 tablet_replica.cc:330] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: stopping tablet replica
W20260809 05:34:49.255152 547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:49.255416 1363 raft_consensus.cc:2229] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:34:49.255496 1361 raft_consensus.cc:2229] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:49.256194 1361 raft_consensus.cc:2258] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:49.256202 1363 raft_consensus.cc:2258] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:49.257311 1362 tablet_replica.cc:330] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: stopping tablet replica
I20260809 05:34:49.257779 1362 raft_consensus.cc:2229] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:49.258307 1362 raft_consensus.cc:2258] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:49.258478 1363 ts_tablet_manager.cc:1905] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:49.260510 1362 ts_tablet_manager.cc:1905] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:49.264945 959 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9f9ec195b46959234e2272e6c0e43 (DEFAULT_TABLE table=test-table [id=532212486bb144cf85172f90c991b161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:49.265113 756 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9f9ec195b46959234e2272e6c0e43 (DEFAULT_TABLE table=test-table [id=532212486bb144cf85172f90c991b161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:49.265450 959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9f9ec195b46959234e2272e6c0e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.265552 756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9f9ec195b46959234e2272e6c0e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.268390 1362 ts_tablet_manager.cc:1918] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:34:49.268623 1362 log.cc:1196] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/481400dadb3e45bd884741f41b7ec9de
I20260809 05:34:49.269287 1362 ts_tablet_manager.cc:1939] T 481400dadb3e45bd884741f41b7ec9de P 66c883e354e74a418c6bc6db44c07173: Deleting consensus metadata
I20260809 05:34:49.271143 1175 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac9f9ec195b46959234e2272e6c0e43 (DEFAULT_TABLE table=test-table [id=532212486bb144cf85172f90c991b161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:49.271301 528 catalog_manager.cc:4841] TS 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683): tablet 481400dadb3e45bd884741f41b7ec9de (table test-table [id=86637a575c034aaca43a285a3d031628]) successfully deleted
I20260809 05:34:49.271595 1175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac9f9ec195b46959234e2272e6c0e43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.272286 1326 tablet_bootstrap.cc:492] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Bootstrap starting.
I20260809 05:34:49.272712 1327 tablet_bootstrap.cc:492] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Bootstrap starting.
I20260809 05:34:49.273626 1361 ts_tablet_manager.cc:1905] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:49.274106 1326 tablet_bootstrap.cc:654] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.274313 1327 tablet_bootstrap.cc:654] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.281639 1361 ts_tablet_manager.cc:1918] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:34:49.281802 1361 log.cc:1196] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/481400dadb3e45bd884741f41b7ec9de
I20260809 05:34:49.282408 1361 ts_tablet_manager.cc:1939] T 481400dadb3e45bd884741f41b7ec9de P 86a082d448e1458ca1fd577b51893f0e: Deleting consensus metadata
I20260809 05:34:49.284850 1363 ts_tablet_manager.cc:1918] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:34:49.285032 1363 log.cc:1196] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/481400dadb3e45bd884741f41b7ec9de
I20260809 05:34:49.285609 1363 ts_tablet_manager.cc:1939] T 481400dadb3e45bd884741f41b7ec9de P 3e27a4542ada44b1a280020562851455: Deleting consensus metadata
I20260809 05:34:49.287860 1327 tablet_bootstrap.cc:492] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: No bootstrap required, opened a new log
I20260809 05:34:49.288039 1327 ts_tablet_manager.cc:1397] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.006s
I20260809 05:34:49.288266 1325 tablet_bootstrap.cc:492] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Bootstrap starting.
I20260809 05:34:49.288414 1326 tablet_bootstrap.cc:492] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: No bootstrap required, opened a new log
I20260809 05:34:49.288545 1326 ts_tablet_manager.cc:1397] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.006s
I20260809 05:34:49.288923 1327 raft_consensus.cc:348] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.289162 1327 raft_consensus.cc:374] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.289264 1327 raft_consensus.cc:729] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e27a4542ada44b1a280020562851455, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.289399 1326 raft_consensus.cc:348] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.289609 1326 raft_consensus.cc:374] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.289700 1326 raft_consensus.cc:729] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a082d448e1458ca1fd577b51893f0e, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.289564 1327 consensus_queue.cc:260] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.290007 1325 tablet_bootstrap.cc:654] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.290153 1327 ts_tablet_manager.cc:1428] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 05:34:49.290032 1326 consensus_queue.cc:260] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.292153 1326 ts_tablet_manager.cc:1428] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 05:34:49.294569 524 catalog_manager.cc:4841] TS 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385): tablet 481400dadb3e45bd884741f41b7ec9de (table test-table [id=86637a575c034aaca43a285a3d031628]) successfully deleted
I20260809 05:34:49.295598 524 catalog_manager.cc:4841] TS 3e27a4542ada44b1a280020562851455 (127.30.137.193:34191): tablet 481400dadb3e45bd884741f41b7ec9de (table test-table [id=86637a575c034aaca43a285a3d031628]) successfully deleted
I20260809 05:34:49.297703 1325 tablet_bootstrap.cc:492] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: No bootstrap required, opened a new log
I20260809 05:34:49.297907 1325 ts_tablet_manager.cc:1397] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
W20260809 05:34:49.298121 858 tablet.cc:2369] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:49.298571 1325 raft_consensus.cc:348] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.298867 1325 raft_consensus.cc:374] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.298974 1325 raft_consensus.cc:729] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c883e354e74a418c6bc6db44c07173, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.299274 1325 consensus_queue.cc:260] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.299851 1325 ts_tablet_manager.cc:1428] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 05:34:49.424649 1331 raft_consensus.cc:482] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:49.424949 1331 raft_consensus.cc:504] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.426982 1331 leader_election.cc:288] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e27a4542ada44b1a280020562851455 (127.30.137.193:34191), 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
I20260809 05:34:49.435635 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.427486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:54423 (local address 127.30.137.195:44683)
0809 05:34:49.427708 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.427715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.428434 (+ 719us) server_negotiation.cc:407] Connection header received
0809 05:34:49.428560 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.428566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.428649 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.428757 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:49.429541 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.430719 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.433572 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.434510 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.434740 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.434822 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.435200 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.435357 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:34:49.436461 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.427319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:44683 (local address 127.30.137.194:54423)
0809 05:34:49.428314 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.428331 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.428443 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.428814 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.428819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.428849 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:49.429309 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.429329 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.430870 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.430877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.433338 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.433355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.434775 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.434781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.434988 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.435023 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.435876 (+ 853us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.436039 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"mutex_wait_us":68,"thread_start_us":406,"threads_started":1}
I20260809 05:34:49.436504 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.427854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:56739 (local address 127.30.137.193:34191)
0809 05:34:49.428012 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.428017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.428219 (+ 202us) server_negotiation.cc:407] Connection header received
0809 05:34:49.428563 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.428571 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.428637 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.428689 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:49.429647 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.430457 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.433534 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.434958 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.435121 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.435197 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.436000 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.436152 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 05:34:49.437062 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.427243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:34191 (local address 127.30.137.194:56739)
0809 05:34:49.428073 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.428099 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.428369 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.428935 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.428940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.428981 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:49.429470 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.429483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.430616 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.430624 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.433342 (+ 2718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.433355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.435375 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.435382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.435567 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.435602 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.436515 (+ 913us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.436679 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":98,"thread_start_us":830,"threads_started":1}
I20260809 05:34:49.437156 1213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac9f9ec195b46959234e2272e6c0e43" candidate_uuid: "86a082d448e1458ca1fd577b51893f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c883e354e74a418c6bc6db44c07173" is_pre_election: true
I20260809 05:34:49.437485 1213 raft_consensus.cc:2454] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86a082d448e1458ca1fd577b51893f0e in term 0.
I20260809 05:34:49.438078 887 leader_election.cc:302] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c883e354e74a418c6bc6db44c07173, 86a082d448e1458ca1fd577b51893f0e; no voters:
I20260809 05:34:49.438140 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac9f9ec195b46959234e2272e6c0e43" candidate_uuid: "86a082d448e1458ca1fd577b51893f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e27a4542ada44b1a280020562851455" is_pre_election: true
I20260809 05:34:49.438603 1331 raft_consensus.cc:2781] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:49.438755 1331 raft_consensus.cc:482] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:49.438714 789 raft_consensus.cc:2454] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86a082d448e1458ca1fd577b51893f0e in term 0.
I20260809 05:34:49.438849 1331 raft_consensus.cc:3037] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.442258 1331 raft_consensus.cc:504] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.443082 1331 leader_election.cc:288] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [CANDIDATE]: Term 1 election: Requested vote from peers 3e27a4542ada44b1a280020562851455 (127.30.137.193:34191), 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
I20260809 05:34:49.443884 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac9f9ec195b46959234e2272e6c0e43" candidate_uuid: "86a082d448e1458ca1fd577b51893f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e27a4542ada44b1a280020562851455"
I20260809 05:34:49.443951 1213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac9f9ec195b46959234e2272e6c0e43" candidate_uuid: "86a082d448e1458ca1fd577b51893f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c883e354e74a418c6bc6db44c07173"
I20260809 05:34:49.444177 1213 raft_consensus.cc:3037] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.444180 789 raft_consensus.cc:3037] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.447811 1213 raft_consensus.cc:2454] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86a082d448e1458ca1fd577b51893f0e in term 1.
I20260809 05:34:49.448074 789 raft_consensus.cc:2454] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86a082d448e1458ca1fd577b51893f0e in term 1.
I20260809 05:34:49.448505 887 leader_election.cc:302] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c883e354e74a418c6bc6db44c07173, 86a082d448e1458ca1fd577b51893f0e; no voters:
I20260809 05:34:49.448837 1331 raft_consensus.cc:2781] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:49.449397 1331 raft_consensus.cc:686] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 1 LEADER]: Becoming Leader. State: Replica: 86a082d448e1458ca1fd577b51893f0e, State: Running, Role: LEADER
I20260809 05:34:49.449910 1331 consensus_queue.cc:237] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.454057 549 catalog_manager.cc:5495] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 86a082d448e1458ca1fd577b51893f0e (127.30.137.194). New cstate: current_term: 1 leader_uuid: "86a082d448e1458ca1fd577b51893f0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } health_report { overall_health: UNKNOWN } } }
Aug 09 05:34:49 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253686, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:49.525520 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.508979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45149 (local address 127.0.0.1:37414)
0809 05:34:49.509554 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.509582 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.509749 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.510217 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.510223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.511186 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:49.511463 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.511476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.512583 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.512589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.514123 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.514138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.515057 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.515066 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.515261 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.516152 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:49.516186 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:49.519131 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:49.523041 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:49.523052 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:49.523058 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:49.523500 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:49.523934 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:49.523937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:49.523939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:49.524073 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:49.524707 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:49.524713 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:49.524859 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.524995 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.525204 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":271,"threads_started":1}
I20260809 05:34:49.525864 1379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.509046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.30.137.254:45149)
0809 05:34:49.509898 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.509903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.509919 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:34:49.509964 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.509968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.510021 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.510117 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:49.511657 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.512418 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.514269 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.514860 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.514957 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.519337 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:49.519342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:49.519345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:49.519375 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:49.522888 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:49.523635 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:49.523641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:49.523645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:49.523820 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:49.524188 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:49.524193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:49.524197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:49.524577 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:49.524735 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.525132 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.525377 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":648,"threads_started":1}
I20260809 05:34:49.531410 549 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:37414:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 05:34:49.532763 549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:49.540582 756 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed3a59596494f9495c28a758f241082 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96df7e9b03df4ecbbab30de22e9785c7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:34:49.540946 1175 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed3a59596494f9495c28a758f241082 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96df7e9b03df4ecbbab30de22e9785c7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:34:49.541266 756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed3a59596494f9495c28a758f241082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.541144 959 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed3a59596494f9495c28a758f241082 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96df7e9b03df4ecbbab30de22e9785c7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:34:49.541494 1175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed3a59596494f9495c28a758f241082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.541723 959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed3a59596494f9495c28a758f241082. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:49.549508 1325 tablet_bootstrap.cc:492] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: Bootstrap starting.
I20260809 05:34:49.549510 1327 tablet_bootstrap.cc:492] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: Bootstrap starting.
I20260809 05:34:49.551491 1325 tablet_bootstrap.cc:654] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.551498 1327 tablet_bootstrap.cc:654] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.552762 1326 tablet_bootstrap.cc:492] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: Bootstrap starting.
I20260809 05:34:49.555143 1326 tablet_bootstrap.cc:654] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:49.563882 1325 tablet_bootstrap.cc:492] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: No bootstrap required, opened a new log
I20260809 05:34:49.564137 1325 ts_tablet_manager.cc:1397] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.010s
I20260809 05:34:49.564306 1327 tablet_bootstrap.cc:492] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: No bootstrap required, opened a new log
I20260809 05:34:49.564558 1327 ts_tablet_manager.cc:1397] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260809 05:34:49.565766 1327 raft_consensus.cc:348] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.566181 1327 raft_consensus.cc:374] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.566332 1327 raft_consensus.cc:729] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e27a4542ada44b1a280020562851455, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.566573 1326 tablet_bootstrap.cc:492] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: No bootstrap required, opened a new log
I20260809 05:34:49.566500 1325 raft_consensus.cc:348] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.566792 1326 ts_tablet_manager.cc:1397] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260809 05:34:49.566812 1325 raft_consensus.cc:374] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.566892 1325 raft_consensus.cc:729] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c883e354e74a418c6bc6db44c07173, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.567162 1325 consensus_queue.cc:260] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.567126 1327 consensus_queue.cc:260] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.567816 1325 ts_tablet_manager.cc:1428] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 05:34:49.567729 1333 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.567754 1332 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.568064 1333 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.568099 1332 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.569175 1327 ts_tablet_manager.cc:1428] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 05:34:49.569254 1326 raft_consensus.cc:348] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.569530 1326 raft_consensus.cc:374] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:49.569617 1326 raft_consensus.cc:729] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a082d448e1458ca1fd577b51893f0e, State: Initialized, Role: FOLLOWER
I20260809 05:34:49.569918 1326 consensus_queue.cc:260] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.570427 1331 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.570788 1331 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.572337 1326 ts_tablet_manager.cc:1428] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 05:34:49.755431 1333 raft_consensus.cc:482] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:49.755631 1333 raft_consensus.cc:504] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.756274 1333 leader_election.cc:288] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385), 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
I20260809 05:34:49.756907 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed3a59596494f9495c28a758f241082" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a082d448e1458ca1fd577b51893f0e" is_pre_election: true
I20260809 05:34:49.756887 1213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed3a59596494f9495c28a758f241082" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c883e354e74a418c6bc6db44c07173" is_pre_election: true
I20260809 05:34:49.757156 994 raft_consensus.cc:2454] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 0.
I20260809 05:34:49.757181 1213 raft_consensus.cc:2454] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 0.
I20260809 05:34:49.757653 676 leader_election.cc:302] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3e27a4542ada44b1a280020562851455, 66c883e354e74a418c6bc6db44c07173, 86a082d448e1458ca1fd577b51893f0e; no voters:
I20260809 05:34:49.757979 1333 raft_consensus.cc:2781] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:49.758121 1333 raft_consensus.cc:482] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:49.758241 1333 raft_consensus.cc:3037] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.760373 1333 raft_consensus.cc:504] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.760936 1333 leader_election.cc:288] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 election: Requested vote from peers 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385), 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683)
I20260809 05:34:49.761516 1213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed3a59596494f9495c28a758f241082" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c883e354e74a418c6bc6db44c07173"
I20260809 05:34:49.761518 994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed3a59596494f9495c28a758f241082" candidate_uuid: "3e27a4542ada44b1a280020562851455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86a082d448e1458ca1fd577b51893f0e"
I20260809 05:34:49.761751 994 raft_consensus.cc:3037] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.761739 1213 raft_consensus.cc:3037] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:49.763893 1213 raft_consensus.cc:2454] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 1.
I20260809 05:34:49.763906 994 raft_consensus.cc:2454] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e27a4542ada44b1a280020562851455 in term 1.
I20260809 05:34:49.764398 678 leader_election.cc:302] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e27a4542ada44b1a280020562851455, 66c883e354e74a418c6bc6db44c07173; no voters:
I20260809 05:34:49.764739 1333 raft_consensus.cc:2781] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:49.765130 1333 raft_consensus.cc:686] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [term 1 LEADER]: Becoming Leader. State: Replica: 3e27a4542ada44b1a280020562851455, State: Running, Role: LEADER
I20260809 05:34:49.765465 1333 consensus_queue.cc:237] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } }
I20260809 05:34:49.767194 1388 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: TxnStatusTablet state changed. Reason: New leader 3e27a4542ada44b1a280020562851455. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.767627 1388 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:34:49.768435 1390 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 05:34:49.768767 1390 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 05:34:49.769060 549 catalog_manager.cc:5495] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e27a4542ada44b1a280020562851455 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } health_report { overall_health: UNKNOWN } } }
I20260809 05:34:49.814792 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.808021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48278 (local address 127.30.137.193:34191)
0809 05:34:49.808186 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.808191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.808210 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:34:49.808569 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.808577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.808660 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.808764 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:49.809220 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.810433 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.812498 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.813195 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.813324 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.813950 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.814082 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.814193 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.814364 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.814447 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 05:34:49.814990 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.807926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:34191 (local address 127.0.0.1:48278)
0809 05:34:49.808138 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.808160 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.808304 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.808837 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.808842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.808855 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:49.809060 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.809072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.810591 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.810597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.812332 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.812348 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.813436 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.813444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.813676 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.813750 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.814208 (+ 458us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.814234 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.814327 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.814598 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 05:34:49.820441 1213 raft_consensus.cc:1261] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Refusing update from remote peer 3e27a4542ada44b1a280020562851455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:49.820851 994 raft_consensus.cc:1261] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Refusing update from remote peer 3e27a4542ada44b1a280020562851455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:49.821156 1388 consensus_queue.cc:1035] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:49.821463 1333 consensus_queue.cc:1035] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:49.832324 1332 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: TxnStatusTablet state changed. Reason: New leader 3e27a4542ada44b1a280020562851455. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.832656 1332 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.837877 1333 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.839598 1331 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: TxnStatusTablet state changed. Reason: New leader 3e27a4542ada44b1a280020562851455. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.839907 1331 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.844705 1332 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.845003 1332 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 66c883e354e74a418c6bc6db44c07173: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.845232 1333 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:34:49.847396 1333 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.847653 1333 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 3e27a4542ada44b1a280020562851455: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:34:49.850631 1331 tablet_replica.cc:439] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e27a4542ada44b1a280020562851455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 } } peers { permanent_uuid: "86a082d448e1458ca1fd577b51893f0e" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37385 } } peers { permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 } } }
I20260809 05:34:49.851795 1331 tablet_replica.cc:442] T 7ed3a59596494f9495c28a758f241082 P 86a082d448e1458ca1fd577b51893f0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:49.852296 1395 mvcc.cc:205] Tried to move back new op lower bound from 7316495113497350144 to 7316495113281650688. Current Snapshot: MvccSnapshot[applied={T|T < 7316495113497350144 or (T in {7316495113497350144})}]
I20260809 05:34:49.861882 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.854570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.30.137.194:37385)
0809 05:34:49.855455 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.855459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.855474 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:34:49.855550 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.855553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.855622 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.855777 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:49.856374 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.857194 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.859266 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.859720 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.859823 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.860395 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.861073 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.861192 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.861361 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.861432 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":667,"threads_started":1}
I20260809 05:34:49.873102 1331 consensus_queue.cc:1035] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66c883e354e74a418c6bc6db44c07173" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 44683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:49.879839 1369 consensus_queue.cc:1035] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e27a4542ada44b1a280020562851455" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:49.889312 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.874558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:60185 (local address 127.30.137.193:34191)
0809 05:34:49.874814 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.874820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.882977 (+ 8157us) server_negotiation.cc:407] Connection header received
0809 05:34:49.883211 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.883216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.883292 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.883433 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:49.883956 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.884761 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.886218 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.886764 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.886888 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.888370 (+ 1482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.888543 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.888618 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.888888 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.888981 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:34:49.890206 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.874297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:34191 (local address 127.30.137.194:60185)
0809 05:34:49.882845 (+ 8548us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.882862 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.883061 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.883452 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.883457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.883469 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:49.883788 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.883797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.884922 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.884928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.886035 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.886047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.887073 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.887081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.887303 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.887403 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.888733 (+ 1330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.888746 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.889619 (+ 873us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.889795 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8383,"thread_start_us":8221,"threads_started":1}
I20260809 05:34:49.943807 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.936604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:44109 (local address 127.30.137.194:37385)
0809 05:34:49.936818 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:49.936826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:49.937747 (+ 921us) server_negotiation.cc:407] Connection header received
0809 05:34:49.937988 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:49.937993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:49.938073 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:49.938183 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:49.938906 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.939964 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.941845 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.942289 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.942438 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.942993 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.943108 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.943224 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:49.943412 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 05:34:49.943504 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 05:34:49.944962 1420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:49.936863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37385 (local address 127.30.137.193:44109)
0809 05:34:49.937609 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:49.937626 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:49.937829 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:49.938209 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:49.938214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:49.938226 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:49.938653 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.938666 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.940153 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.940158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.941688 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:49.941704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:49.942502 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:49.942509 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:49.942723 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:49.942796 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:49.943224 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:49.943239 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 05:34:49.944349 (+ 1110us) client_negotiation.cc:240] Negotiation successful
0809 05:34:49.944512 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":368,"threads_started":1}
I20260809 05:34:49.992803 547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:49.993139 547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:49.996984 547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e47cc1dca01448b6a6a2cb1216a83c85: Sending DeleteTablet for 3 replicas of tablet 0ac9f9ec195b46959234e2272e6c0e43
I20260809 05:34:49.998229 1175 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9f9ec195b46959234e2272e6c0e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:49 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:35898
I20260809 05:34:49.998348 959 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9f9ec195b46959234e2272e6c0e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:49 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:53244
I20260809 05:34:49.998373 756 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac9f9ec195b46959234e2272e6c0e43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:49 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:48266
I20260809 05:34:50.008832 1424 tablet_replica.cc:330] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: stopping tablet replica
I20260809 05:34:50.009245 1424 raft_consensus.cc:2229] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:50.009689 1424 raft_consensus.cc:2258] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:50.011268 1424 ts_tablet_manager.cc:1905] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:50.011891 1423 tablet_replica.cc:330] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: stopping tablet replica
I20260809 05:34:50.013449 1423 raft_consensus.cc:2229] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:50.013808 1423 raft_consensus.cc:2258] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:50.015177 1423 ts_tablet_manager.cc:1905] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:50.028452 1423 ts_tablet_manager.cc:1918] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:50.030982 1424 ts_tablet_manager.cc:1918] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:50.031219 1424 log.cc:1196] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/0ac9f9ec195b46959234e2272e6c0e43
I20260809 05:34:50.031879 1424 ts_tablet_manager.cc:1939] T 0ac9f9ec195b46959234e2272e6c0e43 P 3e27a4542ada44b1a280020562851455: Deleting consensus metadata
I20260809 05:34:50.032088 1423 log.cc:1196] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/0ac9f9ec195b46959234e2272e6c0e43
I20260809 05:34:50.033422 524 catalog_manager.cc:4841] TS 3e27a4542ada44b1a280020562851455 (127.30.137.193:34191): tablet 0ac9f9ec195b46959234e2272e6c0e43 (table test-table [id=532212486bb144cf85172f90c991b161]) successfully deleted
I20260809 05:34:50.034144 1423 ts_tablet_manager.cc:1939] T 0ac9f9ec195b46959234e2272e6c0e43 P 66c883e354e74a418c6bc6db44c07173: Deleting consensus metadata
Aug 09 05:34:50 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:50.035425 528 catalog_manager.cc:4841] TS 66c883e354e74a418c6bc6db44c07173 (127.30.137.195:44683): tablet 0ac9f9ec195b46959234e2272e6c0e43 (table test-table [id=532212486bb144cf85172f90c991b161]) successfully deleted
I20260809 05:34:50.035804 1427 tablet_replica.cc:330] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: stopping tablet replica
I20260809 05:34:50.036043 1427 raft_consensus.cc:2229] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:34:50.036640 1427 raft_consensus.cc:2258] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:50.037854 1427 ts_tablet_manager.cc:1905] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:50.045351 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:50.015420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.30.137.193:34191)
0809 05:34:50.015682 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:50.015692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:50.016909 (+ 1217us) server_negotiation.cc:407] Connection header received
0809 05:34:50.018102 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:50.018109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:50.018201 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:50.018292 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:50.022774 (+ 4482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.023976 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.028099 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.028845 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.028963 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:50.037312 (+ 8349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:50.037330 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:50.037349 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:50.037406 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:50.041568 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.042517 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.042527 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.042531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.042762 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.043298 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.043303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.043306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.043779 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:50.044094 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:50.044744 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 05:34:50.044983 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 05:34:50.046211 1427 ts_tablet_manager.cc:1918] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:50.046372 1427 log.cc:1196] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/0ac9f9ec195b46959234e2272e6c0e43
I20260809 05:34:50.046983 1427 ts_tablet_manager.cc:1939] T 0ac9f9ec195b46959234e2272e6c0e43 P 86a082d448e1458ca1fd577b51893f0e: Deleting consensus metadata
W20260809 05:34:50.047264 848 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48288
I20260809 05:34:50.048228 524 catalog_manager.cc:4841] TS 86a082d448e1458ca1fd577b51893f0e (127.30.137.194:37385): tablet 0ac9f9ec195b46959234e2272e6c0e43 (table test-table [id=532212486bb144cf85172f90c991b161]) successfully deleted
I20260809 05:34:50.085486 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:50.050538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.30.137.254:45149)
0809 05:34:50.067931 (+ 17393us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:50.067937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:50.067959 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:34:50.068025 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:50.068031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:50.068110 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:50.068228 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:50.070313 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.071779 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.073610 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.074316 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.074450 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:50.077630 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:50.077637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:50.077642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:50.077684 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:50.081455 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.082320 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.082327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.082331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.082483 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.082943 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.082949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.082952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.083354 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:50.083649 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:50.084817 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0809 05:34:50.085064 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17267,"thread_start_us":16491,"threads_started":1}
W20260809 05:34:50.086125 547 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37426
I20260809 05:34:50.092262 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 650
I20260809 05:34:50.139837 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:50.099127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.30.137.193:34191)
0809 05:34:50.099351 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:50.099358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:50.113561 (+ 14203us) server_negotiation.cc:407] Connection header received
0809 05:34:50.114135 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:50.114143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:50.114237 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:50.114319 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:50.117805 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.119086 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.126824 (+ 7738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.127626 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.127764 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:50.131384 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:50.131391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:50.131396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:50.131445 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:50.136033 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.137045 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.137051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.137055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.137240 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.137744 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.137750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.137753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.138173 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:50.138446 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:50.139117 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 05:34:50.139406 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 05:34:50.141409 848 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48300
W20260809 05:34:50.142093 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 650: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:34:50.142306 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 650
I20260809 05:34:50.187990 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 862
Aug 09 05:34:50 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:34:50.204595 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:50.188916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.30.137.194:37385)
0809 05:34:50.189128 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:50.189136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:50.189158 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:34:50.189391 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:50.189397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:50.189480 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:50.189631 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:50.190889 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.191715 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.193031 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.193512 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.193671 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:50.198162 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:50.198178 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:50.198201 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:50.198249 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:50.201559 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.202295 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.202317 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.202319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.202463 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.202968 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.202974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.202977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.203372 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:50.203747 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:50.203970 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 05:34:50.204205 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 05:34:50.205386 1062 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53256
W20260809 05:34:50.206228 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 862: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:34:50.206452 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 862
I20260809 05:34:50.234802 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1077
Aug 09 05:34:50 dist-test-slave-3kk6 krb5kdc[479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253688, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:34:50.250911 1435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:50.235673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.30.137.195:44683)
0809 05:34:50.236218 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:50.236223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:50.236240 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:34:50.236300 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:50.236304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:50.236364 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:50.236460 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:50.237789 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.238512 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.239934 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:50.240399 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:50.240504 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:50.245162 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:50.245179 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:50.245196 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:50.245238 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:50.248211 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.248879 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.248886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.248888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.249021 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:50.249430 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:50.249433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:50.249435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:50.249772 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:50.250004 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:50.250291 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 05:34:50.250473 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":324,"threads_started":1}
W20260809 05:34:50.251768 1284 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35906
W20260809 05:34:50.252468 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1077: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:34:50.252610 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 1077
I20260809 05:34:50.279686 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 501
I20260809 05:34:50.280921 606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:34:50.401633 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 501
2026-08-09T05:34:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5313 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:34:50 dist-test-slave-3kk6 krb5kdc[1441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:34:50 dist-test-slave-3kk6 krb5kdc[1441](info): set up 2 sockets
Aug 09 05:34:50 dist-test-slave-3kk6 krb5kdc[1441](info): commencing operation
krb5kdc: starting...
W20260809 05:34:51.749182 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:34:51 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:34:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:34:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:34:51.976444 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42309
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:42309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:52.319010 1468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:52.319415 1468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:52.319593 1468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:52.327039 1468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:34:52.327239 1468 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 05:34:52.327347 1468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:52.327481 1468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:52.327582 1468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:34:52.327696 1468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:34:52.338065 1468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:42309
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:52.339409 1468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:52.340771 1468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:52.353484 1474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:52.353550 1477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:52.354221 1475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:52.354286 1468 server_base.cc:1034] running on GCE node
I20260809 05:34:52.355300 1468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:52.356925 1468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:52.358198 1468 hybrid_clock.cc:648] HybridClock initialized: now 1786253692358137 us; error 47 us; skew 500 ppm
I20260809 05:34:52.358569 1468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:52 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253692, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:52.362035 1468 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:34:52.363794 1468 webserver.cc:460] Webserver started at http://127.30.137.254:35849/ using document root <none> and password file <none>
I20260809 05:34:52.364565 1468 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:52.364715 1468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:52.365075 1468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:52.368270 1468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6"
format_stamp: "Formatted at 2026-08-09 05:34:52 on dist-test-slave-3kk6"
server_key: "89431ca0dec053a04ef478c746d7bdba"
server_key_iv: "c34bf727e8af46ec6676f1e964aa8e65"
server_key_version: "encryptionkey@0"
I20260809 05:34:52.369124 1468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6"
format_stamp: "Formatted at 2026-08-09 05:34:52 on dist-test-slave-3kk6"
server_key: "89431ca0dec053a04ef478c746d7bdba"
server_key_iv: "c34bf727e8af46ec6676f1e964aa8e65"
server_key_version: "encryptionkey@0"
I20260809 05:34:52.374882 1468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:34:52.378414 1487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:52.379828 1468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:34:52.380038 1468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6"
format_stamp: "Formatted at 2026-08-09 05:34:52 on dist-test-slave-3kk6"
server_key: "89431ca0dec053a04ef478c746d7bdba"
server_key_iv: "c34bf727e8af46ec6676f1e964aa8e65"
server_key_version: "encryptionkey@0"
I20260809 05:34:52.380271 1468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:52.394129 1468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:52.398743 1468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:52.399101 1468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:52.423005 1468 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:42309
I20260809 05:34:52.422966 1569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:42309 every 8 connection(s)
I20260809 05:34:52.424536 1468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:34:52.428903 1570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:52.430395 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 1468
I20260809 05:34:52.430696 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:34:52.431734 31271 external_mini_cluster.cc:1428] Setting key a369368af4ea798a64de52ed6cfd9790
I20260809 05:34:52.444391 1570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Bootstrap starting.
Aug 09 05:34:52 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:52.448365 1570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:52.449869 1570 log.cc:824] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:52.453394 1570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: No bootstrap required, opened a new log
I20260809 05:34:52.456313 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:52.434225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57720 (local address 127.30.137.254:42309)
0809 05:34:52.435107 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:52.435120 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:52.435164 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:34:52.436896 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:52.436925 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:52.437373 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:52.437816 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:52.439478 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:52.440827 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:52.442137 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:52.442964 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:52.443168 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:52.447878 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:52.447887 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:52.447904 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:52.447948 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:52.451883 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:52.452688 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:52.452696 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:52.452703 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:52.452906 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:52.453328 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:52.453334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:52.453337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:52.453996 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:52.454304 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:52.454815 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 05:34:52.455272 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":423,"threads_started":1}
I20260809 05:34:52.462627 1570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42309 } }
I20260809 05:34:52.463207 1570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:52.463358 1570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2bb3b33cc0421c81d148d8ff53b9b6, State: Initialized, Role: FOLLOWER
I20260809 05:34:52.464080 1570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42309 } }
I20260809 05:34:52.464454 1570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:34:52.464610 1570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:34:52.464840 1570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:52.467815 1570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42309 } }
I20260809 05:34:52.468423 1570 leader_election.cc:302] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a2bb3b33cc0421c81d148d8ff53b9b6; no voters:
I20260809 05:34:52.469137 1570 leader_election.cc:288] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:34:52.469321 1577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:52.470149 1577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2bb3b33cc0421c81d148d8ff53b9b6, State: Running, Role: LEADER
I20260809 05:34:52.470798 1577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42309 } }
I20260809 05:34:52.471913 1570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:34:52.474395 1579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a2bb3b33cc0421c81d148d8ff53b9b6. Latest consensus state: current_term: 1 leader_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42309 } } }
I20260809 05:34:52.474866 1579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:52.474375 1578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a2bb3b33cc0421c81d148d8ff53b9b6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42309 } } }
I20260809 05:34:52.475080 1578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6 [sys.catalog]: This master's current role is: LEADER
I20260809 05:34:52.477329 1587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:34:52.484030 1587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:34:52.495263 1587 catalog_manager.cc:1347] Generated new cluster ID: bb80d9b2fdc64b3582d5081398f86963
I20260809 05:34:52.495419 1587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:34:52.533581 1587 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:34:52.535800 1587 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:34:52.563060 1587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Generated new TSK 0
I20260809 05:34:52.564229 1587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:34:52.622511 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42309
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:52.969960 1608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:52.970252 1608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:52.970359 1608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:52.978302 1608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:52.978420 1608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:52.978508 1608 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 05:34:52.978564 1608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:34:52.978628 1608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:34:52.989490 1608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:42309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:52.990928 1608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:52.992264 1608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:53.005800 1618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:53.005730 1620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:53.005859 1617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:53.008169 1608 server_base.cc:1034] running on GCE node
I20260809 05:34:53.008831 1608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:53.009819 1608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:53.011065 1608 hybrid_clock.cc:648] HybridClock initialized: now 1786253693011026 us; error 57 us; skew 500 ppm
I20260809 05:34:53.011404 1608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:53 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253693, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:53.019292 1608 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:34:53.020957 1608 webserver.cc:460] Webserver started at http://127.30.137.193:39361/ using document root <none> and password file <none>
I20260809 05:34:53.021725 1608 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:53.021880 1608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:53.022228 1608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:53.025343 1608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "4728f2a6869e4b028db52b6b1e511aa7"
format_stamp: "Formatted at 2026-08-09 05:34:53 on dist-test-slave-3kk6"
server_key: "cf2154e46b9626f6b173e736b444a6e3"
server_key_iv: "860d8813f3a7d679c2f3dc9e48c474eb"
server_key_version: "encryptionkey@0"
I20260809 05:34:53.026146 1608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "4728f2a6869e4b028db52b6b1e511aa7"
format_stamp: "Formatted at 2026-08-09 05:34:53 on dist-test-slave-3kk6"
server_key: "cf2154e46b9626f6b173e736b444a6e3"
server_key_iv: "860d8813f3a7d679c2f3dc9e48c474eb"
server_key_version: "encryptionkey@0"
I20260809 05:34:53.031685 1608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 05:34:53.035301 1632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:53.036631 1608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:34:53.036819 1608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "4728f2a6869e4b028db52b6b1e511aa7"
format_stamp: "Formatted at 2026-08-09 05:34:53 on dist-test-slave-3kk6"
server_key: "cf2154e46b9626f6b173e736b444a6e3"
server_key_iv: "860d8813f3a7d679c2f3dc9e48c474eb"
server_key_version: "encryptionkey@0"
I20260809 05:34:53.037036 1608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:53.065544 1608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:53.069878 1608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:53.070209 1608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:53.073225 1608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:53.073439 1608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 05:34:53.073591 1608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:53.073698 1608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:34:53 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253693, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:53.097339 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.073936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:58055 (local address 127.30.137.254:42309)
0809 05:34:53.076418 (+ 2482us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:53.076427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:53.076453 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:34:53.078053 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:53.078058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:53.078152 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:53.078282 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:53.080755 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.081517 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.083589 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.084227 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.084388 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.090223 (+ 5835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:53.090232 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:53.090239 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:53.090289 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:53.093532 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.094433 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.094441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.094447 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.094613 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.095295 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.095299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.095311 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.095587 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:53.095747 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:53.096878 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0809 05:34:53.097059 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2330,"thread_start_us":2214,"threads_started":1}
I20260809 05:34:53.099601 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.074591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.30.137.193:58055)
0809 05:34:53.075597 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:53.075655 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:53.077730 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:53.078542 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:53.078551 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:53.079539 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:53.080537 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.080575 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.081682 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.081688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.083394 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.083413 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.084532 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.084541 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.084799 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.085773 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:53.085808 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:53.089871 (+ 4063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:53.093693 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.093709 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.093727 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.094263 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.094784 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.094792 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.094799 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.095146 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.095738 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:53.095750 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:53.096373 (+ 623us) client_negotiation.cc:769] Sending connection context
0809 05:34:53.096774 (+ 401us) client_negotiation.cc:240] Negotiation successful
0809 05:34:53.097530 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":405,"threads_started":1}
I20260809 05:34:53.128569 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.106150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:55969 (local address 127.30.137.254:42309)
0809 05:34:53.106365 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:53.106373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:53.112945 (+ 6572us) server_negotiation.cc:407] Connection header received
0809 05:34:53.113122 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:53.113126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:53.113197 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:53.113322 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:53.115131 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.116219 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.118407 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.119057 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.119157 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.122167 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:53.122175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:53.122180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:53.122222 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:53.125681 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.126437 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.126447 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.126450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.126579 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.127040 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.127045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.127049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.127357 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:53.127516 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:53.127962 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 05:34:53.128187 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:34:53.128674 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.106148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.30.137.193:55969)
0809 05:34:53.112784 (+ 6636us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:53.112825 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:53.113009 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:53.113369 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:53.113375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:53.114338 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:53.114967 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.114980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.116412 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.116415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.118214 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.118230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.119276 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.119285 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.119491 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.120339 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:53.120366 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:53.121932 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:53.125821 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.125831 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.125836 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.126238 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.126718 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.126726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.126731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.126880 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.127479 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:53.127483 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:53.127673 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 05:34:53.127839 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 05:34:53.128184 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6417,"thread_start_us":943,"threads_started":1}
I20260809 05:34:53.133198 1608 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:33255
I20260809 05:34:53.133220 1811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:33255 every 8 connection(s)
I20260809 05:34:53.134521 1608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:34:53.136916 1814 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42309
I20260809 05:34:53.136936 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 1608
I20260809 05:34:53.137164 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:34:53.137190 1814 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:53.138058 31271 external_mini_cluster.cc:1428] Setting key e50b7ece41bc0cdc9b59cd1c9e6e8cc9
I20260809 05:34:53.138046 1814 heartbeater.cc:511] Master 127.30.137.254:42309 requested a full tablet report, sending...
I20260809 05:34:53.140460 1508 ts_manager.cc:194] Registered new tserver with Master: 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255)
I20260809 05:34:53.142746 1508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:58055
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:34:53.187781 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42309
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:53.529834 1822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:53.530179 1822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:53.530370 1822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:53.537738 1822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:53.537926 1822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:53.538096 1822 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 05:34:53.538194 1822 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:34:53.538344 1822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:34:53.548281 1822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:42309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:53.549674 1822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:53.551121 1822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:53.564757 1828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:53.564774 1827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:53.564810 1831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:53.565845 1822 server_base.cc:1034] running on GCE node
I20260809 05:34:53.566522 1822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:53.567541 1822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:53.568787 1822 hybrid_clock.cc:648] HybridClock initialized: now 1786253693568749 us; error 49 us; skew 500 ppm
I20260809 05:34:53.569131 1822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:53 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253693, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:53.572649 1822 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:34:53.574186 1822 webserver.cc:460] Webserver started at http://127.30.137.194:33017/ using document root <none> and password file <none>
I20260809 05:34:53.575004 1822 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:53.575115 1822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:53.575389 1822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:53.578364 1822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "a804b80cfe104ef1bf253de5b5d86318"
format_stamp: "Formatted at 2026-08-09 05:34:53 on dist-test-slave-3kk6"
server_key: "fa32fc3c941d6bf546415dd7b355b392"
server_key_iv: "74127db0630f93ac8efb9b4a7dcb07f2"
server_key_version: "encryptionkey@0"
I20260809 05:34:53.579509 1822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "a804b80cfe104ef1bf253de5b5d86318"
format_stamp: "Formatted at 2026-08-09 05:34:53 on dist-test-slave-3kk6"
server_key: "fa32fc3c941d6bf546415dd7b355b392"
server_key_iv: "74127db0630f93ac8efb9b4a7dcb07f2"
server_key_version: "encryptionkey@0"
I20260809 05:34:53.585217 1822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:34:53.588762 1843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:53.590152 1822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:34:53.590294 1822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "a804b80cfe104ef1bf253de5b5d86318"
format_stamp: "Formatted at 2026-08-09 05:34:53 on dist-test-slave-3kk6"
server_key: "fa32fc3c941d6bf546415dd7b355b392"
server_key_iv: "74127db0630f93ac8efb9b4a7dcb07f2"
server_key_version: "encryptionkey@0"
I20260809 05:34:53.590469 1822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:53.601821 1822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:53.605875 1822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:53.606122 1822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:53.610986 1822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:53.611129 1822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:53.611266 1822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:53.611388 1822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:34:53 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253693, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:53.634763 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.611428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:57693 (local address 127.30.137.254:42309)
0809 05:34:53.611633 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:53.611659 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:53.613118 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 05:34:53.615228 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:53.615236 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:53.615348 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:53.615478 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:53.617919 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.619298 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.620991 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.621795 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.621966 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.627191 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:53.627197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:53.627200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:53.627235 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:53.630970 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.632037 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.632043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.632047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.632186 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.632668 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.632674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.632679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.632992 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:53.633163 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:53.634111 (+ 948us) server_negotiation.cc:299] Negotiation successful
0809 05:34:53.634362 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 05:34:53.636008 1857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.612004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.30.137.194:57693)
0809 05:34:53.612905 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:53.612964 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:53.614944 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:53.615806 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:53.615818 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:53.616802 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:53.617672 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.617709 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.619483 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.619489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.620818 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.620830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.622084 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.622098 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.622340 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.623328 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:53.623354 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:53.626913 (+ 3559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:53.631150 (+ 4237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.631159 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.631176 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.631898 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.632312 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.632317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.632320 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.632543 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.633135 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:53.633144 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:53.633688 (+ 544us) client_negotiation.cc:769] Sending connection context
0809 05:34:53.634007 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 05:34:53.634605 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":419,"threads_started":1}
I20260809 05:34:53.659726 1653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.642489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:38051 (local address 127.30.137.254:42309)
0809 05:34:53.642712 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:53.642718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:53.643433 (+ 715us) server_negotiation.cc:407] Connection header received
0809 05:34:53.643640 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:53.643646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:53.643718 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:53.643822 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:53.644785 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.645550 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.647710 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.648319 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.648397 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.651570 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:53.651575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:53.651579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:53.651610 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:53.656058 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.657325 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.657333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.657339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.657536 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:53.657979 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:53.657985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:53.657988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:53.658461 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:53.658628 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:53.659127 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 05:34:53.659352 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:34:53.659981 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:53.642677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.30.137.194:38051)
0809 05:34:53.643300 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:53.643316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:53.643509 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:53.643852 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:53.643863 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:53.644342 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:53.644635 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.644644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.645707 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.645716 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.647494 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:53.647512 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:53.648718 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:53.648729 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:53.648978 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:53.649838 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:53.649854 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:53.651374 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:53.656242 (+ 4868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.656248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.656255 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.657132 (+ 877us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.657674 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:53.657682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:53.657690 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:53.657837 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:53.658592 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:53.658596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:53.658844 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 05:34:53.659004 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 05:34:53.659408 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":362,"threads_started":1}
I20260809 05:34:53.670470 1822 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:39969
I20260809 05:34:53.670509 2017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:39969 every 8 connection(s)
I20260809 05:34:53.671783 1822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:34:53.673998 2020 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42309
I20260809 05:34:53.674245 2020 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:53.675024 2020 heartbeater.cc:511] Master 127.30.137.254:42309 requested a full tablet report, sending...
I20260809 05:34:53.676430 1508 ts_manager.cc:194] Registered new tserver with Master: a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:53.677186 1508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:57693
I20260809 05:34:53.680657 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 1822
I20260809 05:34:53.680853 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:34:53.681324 31271 external_mini_cluster.cc:1428] Setting key d018d616be3741df6c6b77fd997f99b8
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:34:53.730602 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42309
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:34:54.075419 2025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:34:54.075790 2025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:34:54.075974 2025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:34:54.083683 2025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:34:54.083868 2025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:34:54.084053 2025 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 05:34:54.084167 2025 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:34:54.084251 2025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:34:54.095010 2025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:42309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:34:54.097121 2025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:34:54.099225 2025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:34:54.115285 2036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:54.115353 2033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:34:54.115228 2032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:34:54.116473 2025 server_base.cc:1034] running on GCE node
I20260809 05:34:54.117125 2025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:34:54.118172 2025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:34:54.119422 2025 hybrid_clock.cc:648] HybridClock initialized: now 1786253694119360 us; error 59 us; skew 500 ppm
I20260809 05:34:54.119761 2025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:34:54 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253694, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:34:54.123119 2025 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:34:54.124686 2025 webserver.cc:460] Webserver started at http://127.30.137.195:42147/ using document root <none> and password file <none>
I20260809 05:34:54.125404 2025 fs_manager.cc:359] Metadata directory not provided
I20260809 05:34:54.125564 2025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:34:54.125902 2025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:34:54.129058 2025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "95d1e6993b274ab0b7fab9b74c20b8d2"
format_stamp: "Formatted at 2026-08-09 05:34:54 on dist-test-slave-3kk6"
server_key: "555008c6802a2666ca6904ee762c8fb8"
server_key_iv: "f119ee4a95c252e77ebffe53c96e6c72"
server_key_version: "encryptionkey@0"
I20260809 05:34:54.129884 2025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "95d1e6993b274ab0b7fab9b74c20b8d2"
format_stamp: "Formatted at 2026-08-09 05:34:54 on dist-test-slave-3kk6"
server_key: "555008c6802a2666ca6904ee762c8fb8"
server_key_iv: "f119ee4a95c252e77ebffe53c96e6c72"
server_key_version: "encryptionkey@0"
I20260809 05:34:54.135803 2025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:34:54.140197 2045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:54.141696 2025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 05:34:54.141913 2025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "95d1e6993b274ab0b7fab9b74c20b8d2"
format_stamp: "Formatted at 2026-08-09 05:34:54 on dist-test-slave-3kk6"
server_key: "555008c6802a2666ca6904ee762c8fb8"
server_key_iv: "f119ee4a95c252e77ebffe53c96e6c72"
server_key_version: "encryptionkey@0"
I20260809 05:34:54.142141 2025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:34:54.146407 1814 heartbeater.cc:503] Master 127.30.137.254:42309 was elected leader, sending a full tablet report...
I20260809 05:34:54.155964 2025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:34:54.160109 2025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:34:54.160449 2025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:34:54.163331 2025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:34:54.163480 2025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:34:54.163612 2025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:34:54.163715 2025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:34:54 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253694, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:54.189046 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.163949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:34793 (local address 127.30.137.254:42309)
0809 05:34:54.164708 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.164714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.165938 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 05:34:54.169294 (+ 3356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.169302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.169421 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.169556 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:54.172012 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.173009 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.175845 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.176519 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.176668 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.181711 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:54.181716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:54.181719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:54.181753 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:54.185472 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.186261 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.186266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.186270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.186413 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.186991 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.186996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.186999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.187422 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:54.187601 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.188377 (+ 776us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.188617 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":511,"threads_started":1}
I20260809 05:34:54.190393 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.164744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.30.137.195:34793)
0809 05:34:54.165714 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.165779 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.168816 (+ 3037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.169835 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.169844 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.170746 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:54.171796 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.171820 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.173184 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.173191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.175662 (+ 2471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.175676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.176792 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.176801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.177019 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.177779 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:54.177802 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:54.181459 (+ 3657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:54.185626 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:54.185635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:54.185653 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:54.186120 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:54.186543 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:54.186548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:54.186552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:54.186854 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:54.187587 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:54.187595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:54.187977 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.188260 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.188829 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":424,"threads_started":1}
I20260809 05:34:54.220639 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.203143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.30.137.195:47653)
0809 05:34:54.203884 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.203901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.204101 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.204597 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.204604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.205420 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:54.205931 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.205950 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.207118 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.207123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.208199 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.208211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.209309 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.209319 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.209598 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.210393 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:54.210425 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:54.212056 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:54.216792 (+ 4736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:54.216798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:54.216802 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:54.217467 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:54.218035 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:54.218041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:54.218045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:54.218261 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:54.219157 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:54.219164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:54.219402 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.219600 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.220021 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":457,"threads_started":1}
I20260809 05:34:54.221580 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.203061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:47653 (local address 127.30.137.254:42309)
0809 05:34:54.203272 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.203278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.204018 (+ 740us) server_negotiation.cc:407] Connection header received
0809 05:34:54.204263 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.204268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.204394 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.204515 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:54.206134 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.206954 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.208370 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.209043 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.209199 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.212403 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:54.212412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:54.212417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:54.212463 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:54.216618 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.217686 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.217693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.217696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.217880 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.218422 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.218430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.218435 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.218862 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:54.219079 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.221003 (+ 1924us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.221283 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:34:54.228001 2025 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:35591
I20260809 05:34:54.228044 2227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:35591 every 8 connection(s)
I20260809 05:34:54.229290 2025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:34:54.231637 2228 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42309
I20260809 05:34:54.231907 2228 heartbeater.cc:464] Registering TS with master...
I20260809 05:34:54.232580 2228 heartbeater.cc:511] Master 127.30.137.254:42309 requested a full tablet report, sending...
I20260809 05:34:54.234063 1508 ts_manager.cc:194] Registered new tserver with Master: 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591)
I20260809 05:34:54.234371 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 2025
I20260809 05:34:54.234571 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:34:54.235101 31271 external_mini_cluster.cc:1428] Setting key 7f7a22ecaa000c4ce0432ec45c06a592
I20260809 05:34:54.235077 1508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:34793
I20260809 05:34:54.238243 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 05:34:54 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 05:34:54 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:54.293676 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.278706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.30.137.254:42309)
0809 05:34:54.278975 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.278985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.279246 (+ 261us) server_negotiation.cc:407] Connection header received
0809 05:34:54.279517 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.279521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.279616 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.279740 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:54.281218 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.281951 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.283320 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.283766 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.283943 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.288072 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:54.288083 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:54.288086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:54.288125 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:54.290784 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.291481 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.291487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.291491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.291636 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.292071 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.292076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.292080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.292384 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:54.292742 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.293044 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.293273 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 05:34:54.298290 1508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:34:54.305346 1508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:54.343344 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.318128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33255 (local address 127.0.0.1:55162)
0809 05:34:54.331630 (+ 13502us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.331743 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.332061 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.334527 (+ 2466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.334535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.334584 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:54.335348 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.335391 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.337301 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.337307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.340243 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.340260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.342229 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.342236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.342441 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.342498 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.342624 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.342866 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13042,"mutex_wait_us":136,"thread_start_us":14890,"threads_started":1}
I20260809 05:34:54.343798 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.318353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.30.137.193:33255)
0809 05:34:54.333572 (+ 15219us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.333583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.333617 (+ 34us) server_negotiation.cc:407] Connection header received
0809 05:34:54.333731 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.333739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.334025 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.335109 (+ 1084us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:54.335638 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.337126 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.340468 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.342014 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.342211 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.342397 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.343168 (+ 771us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.343381 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15017,"thread_start_us":15851,"threads_started":1}
I20260809 05:34:54.344012 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.333811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.30.137.194:39969)
0809 05:34:54.334509 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.334518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.335926 (+ 1408us) server_negotiation.cc:407] Connection header received
0809 05:34:54.336250 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.336264 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.336514 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.336683 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:54.337424 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.339021 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.341373 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.342502 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.342710 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.342877 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.343245 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.343423 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":350,"threads_started":1}
I20260809 05:34:54.344650 2257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.334519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:39969 (local address 127.0.0.1:54782)
0809 05:34:54.335786 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.335805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.336044 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.336778 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.336782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.336810 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:54.337214 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.337230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.339193 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.339198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.341184 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.341201 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.342815 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.342822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.343029 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.343066 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.344274 (+ 1208us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.344386 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":2,"thread_start_us":969,"threads_started":1}
I20260809 05:34:54.344691 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.318869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.30.137.195:35591)
0809 05:34:54.333860 (+ 14991us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.333867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.333895 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:34:54.333982 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.333987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.334259 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.334512 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:54.335763 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.337348 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.340396 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.342014 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.342229 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.342397 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.344177 (+ 1780us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.344375 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14822,"thread_start_us":14606,"threads_started":1}
I20260809 05:34:54.345408 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.318370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:35591 (local address 127.0.0.1:49914)
0809 05:34:54.332700 (+ 14330us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.332727 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.332916 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.334622 (+ 1706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.334629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.334691 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:54.335459 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.335480 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.337483 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.337490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.340186 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.340200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.343656 (+ 3456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.343664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.343847 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.343877 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.343955 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.344998 (+ 1043us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14113,"mutex_wait_us":178,"thread_start_us":14037,"threads_started":1}
I20260809 05:34:54.348613 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 65a28537a61b4665b1a429c319aa5432 (DEFAULT_TABLE table=test-table [id=fd1d4f7c20e54b01b3e4fcff90dca327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:54.348349 1710 tablet_service.cc:1467] Processing CreateTablet for tablet 65a28537a61b4665b1a429c319aa5432 (DEFAULT_TABLE table=test-table [id=fd1d4f7c20e54b01b3e4fcff90dca327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:54.349064 2114 tablet_service.cc:1467] Processing CreateTablet for tablet 65a28537a61b4665b1a429c319aa5432 (DEFAULT_TABLE table=test-table [id=fd1d4f7c20e54b01b3e4fcff90dca327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:54.349895 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a28537a61b4665b1a429c319aa5432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:54.349958 1710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a28537a61b4665b1a429c319aa5432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:54.350426 2114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a28537a61b4665b1a429c319aa5432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:54.362325 2258 tablet_bootstrap.cc:492] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Bootstrap starting.
I20260809 05:34:54.365046 2259 tablet_bootstrap.cc:492] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Bootstrap starting.
I20260809 05:34:54.367223 2258 tablet_bootstrap.cc:654] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:54.368582 2258 log.cc:824] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:54.368916 2259 tablet_bootstrap.cc:654] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:54.370242 2259 log.cc:824] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:54.372406 2258 tablet_bootstrap.cc:492] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: No bootstrap required, opened a new log
I20260809 05:34:54.372763 2258 ts_tablet_manager.cc:1397] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 05:34:54.374819 2261 tablet_bootstrap.cc:492] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Bootstrap starting.
I20260809 05:34:54.378324 2261 tablet_bootstrap.cc:654] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:54.378379 2259 tablet_bootstrap.cc:492] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: No bootstrap required, opened a new log
I20260809 05:34:54.378710 2259 ts_tablet_manager.cc:1397] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260809 05:34:54.379679 2261 log.cc:824] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Log is configured to *not* fsync() on all Append() calls
I20260809 05:34:54.385000 2258 raft_consensus.cc:348] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.385481 2258 raft_consensus.cc:374] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:54.385589 2258 raft_consensus.cc:729] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a804b80cfe104ef1bf253de5b5d86318, State: Initialized, Role: FOLLOWER
I20260809 05:34:54.385987 2261 tablet_bootstrap.cc:492] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: No bootstrap required, opened a new log
I20260809 05:34:54.386305 2261 ts_tablet_manager.cc:1397] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.007s
I20260809 05:34:54.386327 2258 consensus_queue.cc:260] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.390941 2259 raft_consensus.cc:348] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.391426 2259 raft_consensus.cc:374] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:54.391438 2020 heartbeater.cc:503] Master 127.30.137.254:42309 was elected leader, sending a full tablet report...
I20260809 05:34:54.391547 2259 raft_consensus.cc:729] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Initialized, Role: FOLLOWER
I20260809 05:34:54.392207 2259 consensus_queue.cc:260] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.394253 2259 ts_tablet_manager.cc:1428] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260809 05:34:54.395313 2261 raft_consensus.cc:348] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.395720 2261 raft_consensus.cc:374] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:54.395852 2261 raft_consensus.cc:729] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Initialized, Role: FOLLOWER
I20260809 05:34:54.396512 2261 consensus_queue.cc:260] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.397282 2258 ts_tablet_manager.cc:1428] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Time spent starting tablet: real 0.024s user 0.012s sys 0.008s
I20260809 05:34:54.397701 2228 heartbeater.cc:503] Master 127.30.137.254:42309 was elected leader, sending a full tablet report...
I20260809 05:34:54.399099 2261 ts_tablet_manager.cc:1428] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
W20260809 05:34:54.424028 2021 tablet.cc:2369] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:34:54.481060 2231 tablet.cc:2369] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:54.494122 2267 raft_consensus.cc:482] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:54.494562 2267 raft_consensus.cc:504] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.496534 2267 leader_election.cc:288] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:54.504020 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.496967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:37187 (local address 127.30.137.195:35591)
0809 05:34:54.497205 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.497213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.498153 (+ 940us) server_negotiation.cc:407] Connection header received
0809 05:34:54.498358 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.498362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.498423 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.498530 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:54.499239 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.499997 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.501985 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.502946 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.503149 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.503248 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.503574 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.503721 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 05:34:54.504032 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.497048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:51187 (local address 127.30.137.194:39969)
0809 05:34:54.497293 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.497301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.498274 (+ 973us) server_negotiation.cc:407] Connection header received
0809 05:34:54.498495 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.498500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.498563 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.498687 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:54.499374 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.500123 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.501941 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.502908 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.503116 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.503198 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.503580 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.503725 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:34:54.504371 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.497096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:39969 (local address 127.30.137.193:51187)
0809 05:34:54.498145 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.498167 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.498318 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.498723 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.498728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.498770 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:54.499195 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.499205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.500279 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.500284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.501760 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.501773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.503142 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.503196 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.503378 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.503432 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.503548 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.503731 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":55,"thread_start_us":608,"threads_started":1}
I20260809 05:34:54.504503 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.497007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:35591 (local address 127.30.137.193:37187)
0809 05:34:54.498021 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.498038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.498184 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.498549 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.498554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.498577 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:54.499060 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.499071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.500144 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.500152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.501796 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.501809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.503234 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.503240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.503388 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.503425 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.503543 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.503732 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":57,"thread_start_us":621,"threads_started":1}
I20260809 05:34:54.505465 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a28537a61b4665b1a429c319aa5432" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318" is_pre_election: true
I20260809 05:34:54.505461 2149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a28537a61b4665b1a429c319aa5432" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" is_pre_election: true
I20260809 05:34:54.505980 2149 raft_consensus.cc:2454] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 0.
I20260809 05:34:54.505980 1949 raft_consensus.cc:2454] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 0.
I20260809 05:34:54.506735 1642 leader_election.cc:302] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, 95d1e6993b274ab0b7fab9b74c20b8d2; no voters:
I20260809 05:34:54.507172 2267 raft_consensus.cc:2781] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:54.507339 2267 raft_consensus.cc:482] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:54.507457 2267 raft_consensus.cc:3037] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:54.509717 2267 raft_consensus.cc:504] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.510381 2267 leader_election.cc:288] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 election: Requested vote from peers 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:54.510900 2149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a28537a61b4665b1a429c319aa5432" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2"
I20260809 05:34:54.510983 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a28537a61b4665b1a429c319aa5432" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318"
I20260809 05:34:54.511117 2149 raft_consensus.cc:3037] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:54.511226 1949 raft_consensus.cc:3037] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:54.513532 1949 raft_consensus.cc:2454] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 1.
I20260809 05:34:54.513540 2149 raft_consensus.cc:2454] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 1.
I20260809 05:34:54.514135 1639 leader_election.cc:302] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, 95d1e6993b274ab0b7fab9b74c20b8d2, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:54.514499 2267 raft_consensus.cc:2781] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:54.515023 2267 raft_consensus.cc:686] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Running, Role: LEADER
I20260809 05:34:54.515769 2267 consensus_queue.cc:237] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } }
I20260809 05:34:54.520905 1508 catalog_manager.cc:5495] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } health_report { overall_health: UNKNOWN } } }
Aug 09 05:34:54 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253692, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:54.567910 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.553477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57742 (local address 127.30.137.254:42309)
0809 05:34:54.553723 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.553731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.554406 (+ 675us) server_negotiation.cc:407] Connection header received
0809 05:34:54.554561 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.554565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.554629 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.554799 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:54.556134 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.556919 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.558279 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.558741 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.558836 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.562340 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:54.562345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:54.562348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:54.562383 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:54.564934 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.565690 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.565699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.565702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.565826 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:54.566230 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:54.566237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:54.566240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:54.566561 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:54.566737 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.567166 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.567557 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":190,"server-negotiator.queue_time_us":95}
I20260809 05:34:54.567936 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.553507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42309 (local address 127.0.0.1:57742)
0809 05:34:54.554277 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.554296 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.554447 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.554879 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.554885 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.555687 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:34:54.555981 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.555990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.557081 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.557086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.558120 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.558133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.558936 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.558941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.559069 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.559862 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:34:54.559880 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:34:54.562144 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:34:54.565081 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:54.565094 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:54.565102 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:54.565565 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:54.565955 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:34:54.565960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:34:54.565964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:34:54.566118 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:34:54.566728 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:34:54.566736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:34:54.566916 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.567091 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.567440 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":1,"thread_start_us":374,"threads_started":1}
I20260809 05:34:54.573273 1508 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:57742:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 05:34:54.574244 1508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:54.581408 1710 tablet_service.cc:1467] Processing CreateTablet for tablet 00ac78cfa7e04636b5de647f15368b5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09e8230a6db846b9b35a63a7e25838ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:34:54.581732 2114 tablet_service.cc:1467] Processing CreateTablet for tablet 00ac78cfa7e04636b5de647f15368b5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09e8230a6db846b9b35a63a7e25838ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:34:54.581764 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 00ac78cfa7e04636b5de647f15368b5b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09e8230a6db846b9b35a63a7e25838ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:34:54.582540 1710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ac78cfa7e04636b5de647f15368b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:54.582540 2114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ac78cfa7e04636b5de647f15368b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:54.582623 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00ac78cfa7e04636b5de647f15368b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:54.589178 2258 tablet_bootstrap.cc:492] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: Bootstrap starting.
I20260809 05:34:54.590004 2259 tablet_bootstrap.cc:492] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: Bootstrap starting.
I20260809 05:34:54.591076 2258 tablet_bootstrap.cc:654] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:54.591777 2259 tablet_bootstrap.cc:654] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:54.592473 2261 tablet_bootstrap.cc:492] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: Bootstrap starting.
I20260809 05:34:54.593554 2258 tablet_bootstrap.cc:492] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: No bootstrap required, opened a new log
I20260809 05:34:54.593767 2258 ts_tablet_manager.cc:1397] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 05:34:54.594545 2258 raft_consensus.cc:348] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.594852 2261 tablet_bootstrap.cc:654] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:54.594883 2258 raft_consensus.cc:374] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:54.594978 2258 raft_consensus.cc:729] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a804b80cfe104ef1bf253de5b5d86318, State: Initialized, Role: FOLLOWER
I20260809 05:34:54.595201 2258 consensus_queue.cc:260] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.595921 2258 ts_tablet_manager.cc:1428] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 05:34:54.595742 2265 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.596184 2265 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.598942 2261 tablet_bootstrap.cc:492] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: No bootstrap required, opened a new log
I20260809 05:34:54.599098 2261 ts_tablet_manager.cc:1397] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260809 05:34:54.599088 2259 tablet_bootstrap.cc:492] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: No bootstrap required, opened a new log
I20260809 05:34:54.599258 2259 ts_tablet_manager.cc:1397] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 05:34:54.599849 2261 raft_consensus.cc:348] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.600101 2261 raft_consensus.cc:374] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:54.600231 2261 raft_consensus.cc:729] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Initialized, Role: FOLLOWER
I20260809 05:34:54.600159 2259 raft_consensus.cc:348] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.600430 2259 raft_consensus.cc:374] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:54.600531 2259 raft_consensus.cc:729] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Initialized, Role: FOLLOWER
I20260809 05:34:54.600512 2261 consensus_queue.cc:260] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.600800 2259 consensus_queue.cc:260] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.601200 2261 ts_tablet_manager.cc:1428] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 05:34:54.601034 2268 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.601212 2267 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.601425 2268 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.601516 2267 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.601776 2259 ts_tablet_manager.cc:1428] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260809 05:34:54.637384 1816 tablet.cc:2369] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:34:54.856726 2267 raft_consensus.cc:482] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:54.856979 2267 raft_consensus.cc:504] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.857695 2267 leader_election.cc:288] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969), 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591)
I20260809 05:34:54.858278 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ac78cfa7e04636b5de647f15368b5b" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318" is_pre_election: true
I20260809 05:34:54.858349 2149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ac78cfa7e04636b5de647f15368b5b" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" is_pre_election: true
I20260809 05:34:54.858577 1949 raft_consensus.cc:2454] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 0.
I20260809 05:34:54.858603 2149 raft_consensus.cc:2454] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 0.
I20260809 05:34:54.859076 1639 leader_election.cc:302] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:54.859418 2267 raft_consensus.cc:2781] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:54.859506 2267 raft_consensus.cc:482] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:54.859586 2267 raft_consensus.cc:3037] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:54.861872 2267 raft_consensus.cc:504] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.862545 2267 leader_election.cc:288] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 election: Requested vote from peers a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969), 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591)
I20260809 05:34:54.863118 2149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ac78cfa7e04636b5de647f15368b5b" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2"
I20260809 05:34:54.863118 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00ac78cfa7e04636b5de647f15368b5b" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318"
I20260809 05:34:54.863382 1949 raft_consensus.cc:3037] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:54.863382 2149 raft_consensus.cc:3037] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:54.866427 1949 raft_consensus.cc:2454] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 1.
I20260809 05:34:54.866427 2149 raft_consensus.cc:2454] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 1.
I20260809 05:34:54.866943 1642 leader_election.cc:302] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, 95d1e6993b274ab0b7fab9b74c20b8d2; no voters:
I20260809 05:34:54.867308 2267 raft_consensus.cc:2781] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:54.867487 2267 raft_consensus.cc:686] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Running, Role: LEADER
I20260809 05:34:54.867766 2267 consensus_queue.cc:237] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:54.869417 2277 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: TxnStatusTablet state changed. Reason: New leader 4728f2a6869e4b028db52b6b1e511aa7. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.869733 2277 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:34:54.870486 2296 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 05:34:54.870797 2296 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 05:34:54.871042 1505 catalog_manager.cc:5495] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } health_report { overall_health: UNKNOWN } } }
I20260809 05:34:54.895555 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.887894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33255 (local address 127.0.0.1:55174)
0809 05:34:54.888135 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.888160 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.888341 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.889372 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.889376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.889385 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:54.889539 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.889546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.891186 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.891189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.892265 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.892279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.893456 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.893461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:54.893654 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.893719 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:54.894965 (+ 1246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:54.894988 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:34:54.895115 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 05:34:54.895303 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 05:34:54.895697 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.888010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.30.137.193:33255)
0809 05:34:54.888910 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.888916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.888942 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:34:54.889053 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.889059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.889158 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.889333 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:54.889736 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.891055 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.892422 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.893264 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.893391 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.893972 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:54.894826 (+ 854us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:54.894984 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.895188 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.895291 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":564,"threads_started":1}
I20260809 05:34:54.905160 1949 raft_consensus.cc:1261] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Refusing update from remote peer 4728f2a6869e4b028db52b6b1e511aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:54.905160 2149 raft_consensus.cc:1261] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Refusing update from remote peer 4728f2a6869e4b028db52b6b1e511aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:54.906417 2277 consensus_queue.cc:1035] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:54.906821 2267 consensus_queue.cc:1035] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:54.921967 2265 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: TxnStatusTablet state changed. Reason: New leader 4728f2a6869e4b028db52b6b1e511aa7. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.922293 2265 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.925673 2302 mvcc.cc:205] Tried to move back new op lower bound from 7316495134320181248 to 7316495134180454400. Current Snapshot: MvccSnapshot[applied={T|T < 7316495134320181248}]
I20260809 05:34:54.932433 2265 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.932390 2268 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: TxnStatusTablet state changed. Reason: New leader 4728f2a6869e4b028db52b6b1e511aa7. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.932678 2265 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P a804b80cfe104ef1bf253de5b5d86318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.932678 2268 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.931020 2303 mvcc.cc:205] Tried to move back new op lower bound from 7316495134320181248 to 7316495134180454400. Current Snapshot: MvccSnapshot[applied={T|T < 7316495134320181248}]
I20260809 05:34:54.934892 2305 mvcc.cc:205] Tried to move back new op lower bound from 7316495134320181248 to 7316495134180454400. Current Snapshot: MvccSnapshot[applied={T|T < 7316495134320181248}]
I20260809 05:34:54.938462 2277 consensus_queue.cc:1035] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:54.949608 2319 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.950047 2319 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:34:54.951297 2319 consensus_queue.cc:1035] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:54.953069 2268 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.953328 2268 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 95d1e6993b274ab0b7fab9b74c20b8d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:34:54.956300 2267 tablet_replica.cc:439] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } } }
I20260809 05:34:54.956694 2267 tablet_replica.cc:442] T 00ac78cfa7e04636b5de647f15368b5b P 4728f2a6869e4b028db52b6b1e511aa7: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:34:54.983227 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.973992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55182 (local address 127.30.137.193:33255)
0809 05:34:54.974536 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.974545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.975413 (+ 868us) server_negotiation.cc:407] Connection header received
0809 05:34:54.975969 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.975975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.976073 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.976144 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:54.976939 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.978168 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.980485 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:54.981015 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:54.981149 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:54.981765 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:54.981904 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:54.981983 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:54.982178 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 05:34:54.982253 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:34:55.004017 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.997256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:40161 (local address 127.30.137.193:33255)
0809 05:34:54.997521 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:54.997528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:54.997818 (+ 290us) server_negotiation.cc:407] Connection header received
0809 05:34:54.998000 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:54.998006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:54.998136 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:54.998258 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:54.998834 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.000037 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.001679 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.002380 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.002576 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:55.003167 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:55.003303 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:55.003403 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:55.003538 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 05:34:55.003635 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:34:55.004169 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:54.996903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33255 (local address 127.30.137.193:40161)
0809 05:34:54.997702 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:54.997723 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:54.997853 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:54.998273 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:54.998276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:54.998285 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:54.998622 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:54.998629 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.000187 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.000191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:55.001536 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:55.001550 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.002617 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.002621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:55.002860 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:55.002937 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:55.003419 (+ 482us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:55.003428 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:34:55.003482 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 05:34:55.003693 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":407,"threads_started":1}
I20260809 05:34:55.069757 1505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:55.070043 1505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:55.073711 1505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Sending DeleteTablet for 3 replicas of tablet 65a28537a61b4665b1a429c319aa5432
I20260809 05:34:55.075011 2114 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a28537a61b4665b1a429c319aa5432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:55 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:49914
I20260809 05:34:55.075973 2334 tablet_replica.cc:330] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: stopping tablet replica
I20260809 05:34:55.075875 1710 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a28537a61b4665b1a429c319aa5432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:55 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260809 05:34:55.076448 2334 raft_consensus.cc:2229] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:55.076869 2334 raft_consensus.cc:2258] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:55.077057 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a28537a61b4665b1a429c319aa5432 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:55 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54782
I20260809 05:34:55.078495 1505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 05:34:55.079192 2334 ts_tablet_manager.cc:1905] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 05:34:55.079387 1505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:55.089165 2337 tablet_replica.cc:330] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: stopping tablet replica
I20260809 05:34:55.089527 2114 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4ddbaaa5d8475d9384c87fd424b399 (DEFAULT_TABLE table=test-table [id=55c43a0100d74ba29cc42e5fc71e5699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:55.089685 2337 raft_consensus.cc:2229] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:55.090018 2114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4ddbaaa5d8475d9384c87fd424b399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:55.090111 2337 raft_consensus.cc:2258] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:55.090104 1709 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4ddbaaa5d8475d9384c87fd424b399 (DEFAULT_TABLE table=test-table [id=55c43a0100d74ba29cc42e5fc71e5699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:55.090713 1709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4ddbaaa5d8475d9384c87fd424b399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:55.095674 2337 ts_tablet_manager.cc:1905] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:55.089483 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4ddbaaa5d8475d9384c87fd424b399 (DEFAULT_TABLE table=test-table [id=55c43a0100d74ba29cc42e5fc71e5699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:55.097186 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4ddbaaa5d8475d9384c87fd424b399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:55.097949 2261 tablet_bootstrap.cc:492] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Bootstrap starting.
I20260809 05:34:55.100239 2261 tablet_bootstrap.cc:654] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:55.101991 2259 tablet_bootstrap.cc:492] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Bootstrap starting.
I20260809 05:34:55.103590 2261 tablet_bootstrap.cc:492] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: No bootstrap required, opened a new log
I20260809 05:34:55.103760 2261 ts_tablet_manager.cc:1397] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 05:34:55.103821 2259 tablet_bootstrap.cc:654] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:55.104285 2335 tablet_replica.cc:330] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: stopping tablet replica
I20260809 05:34:55.104566 2261 raft_consensus.cc:348] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.104835 2261 raft_consensus.cc:374] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:55.104923 2261 raft_consensus.cc:729] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Initialized, Role: FOLLOWER
I20260809 05:34:55.105068 2335 raft_consensus.cc:2229] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:34:55.105221 2261 consensus_queue.cc:260] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.106038 2335 raft_consensus.cc:2258] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:55.107712 2337 ts_tablet_manager.cc:1918] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:55.107913 2337 log.cc:1196] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/65a28537a61b4665b1a429c319aa5432
I20260809 05:34:55.107952 2261 ts_tablet_manager.cc:1428] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 05:34:55.108497 2337 ts_tablet_manager.cc:1939] T 65a28537a61b4665b1a429c319aa5432 P a804b80cfe104ef1bf253de5b5d86318: Deleting consensus metadata
I20260809 05:34:55.110592 1489 catalog_manager.cc:4841] TS a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969): tablet 65a28537a61b4665b1a429c319aa5432 (table test-table [id=fd1d4f7c20e54b01b3e4fcff90dca327]) successfully deleted
I20260809 05:34:55.111773 2334 ts_tablet_manager.cc:1918] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:55.111968 2334 log.cc:1196] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/65a28537a61b4665b1a429c319aa5432
I20260809 05:34:55.112375 2335 ts_tablet_manager.cc:1905] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:55.112550 2334 ts_tablet_manager.cc:1939] T 65a28537a61b4665b1a429c319aa5432 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting consensus metadata
I20260809 05:34:55.114313 1490 catalog_manager.cc:4841] TS 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591): tablet 65a28537a61b4665b1a429c319aa5432 (table test-table [id=fd1d4f7c20e54b01b3e4fcff90dca327]) successfully deleted
I20260809 05:34:55.114559 2341 tablet_bootstrap.cc:492] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Bootstrap starting.
I20260809 05:34:55.117149 2341 tablet_bootstrap.cc:654] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:55.119081 2335 ts_tablet_manager.cc:1918] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:55.119273 2335 log.cc:1196] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/65a28537a61b4665b1a429c319aa5432
I20260809 05:34:55.119787 2335 ts_tablet_manager.cc:1939] T 65a28537a61b4665b1a429c319aa5432 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting consensus metadata
I20260809 05:34:55.120007 2341 tablet_bootstrap.cc:492] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: No bootstrap required, opened a new log
I20260809 05:34:55.120163 2341 ts_tablet_manager.cc:1397] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 05:34:55.120956 2341 raft_consensus.cc:348] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.121214 2341 raft_consensus.cc:374] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:55.121306 2341 raft_consensus.cc:729] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a804b80cfe104ef1bf253de5b5d86318, State: Initialized, Role: FOLLOWER
I20260809 05:34:55.121387 1489 catalog_manager.cc:4841] TS 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255): tablet 65a28537a61b4665b1a429c319aa5432 (table test-table [id=fd1d4f7c20e54b01b3e4fcff90dca327]) successfully deleted
I20260809 05:34:55.121562 2341 consensus_queue.cc:260] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.122200 2341 ts_tablet_manager.cc:1428] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 05:34:55.126082 2259 tablet_bootstrap.cc:492] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: No bootstrap required, opened a new log
I20260809 05:34:55.126292 2259 ts_tablet_manager.cc:1397] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.001s
I20260809 05:34:55.127054 2259 raft_consensus.cc:348] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.127279 2259 raft_consensus.cc:374] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:55.127403 2259 raft_consensus.cc:729] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Initialized, Role: FOLLOWER
I20260809 05:34:55.127677 2259 consensus_queue.cc:260] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.128288 2259 ts_tablet_manager.cc:1428] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 05:34:55.372381 2322 raft_consensus.cc:482] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:55.372704 2322 raft_consensus.cc:504] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.374449 2322 leader_election.cc:288] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:55.378753 2319 raft_consensus.cc:482] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:55.379043 2319 raft_consensus.cc:504] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.379900 2319 leader_election.cc:288] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969), 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591)
I20260809 05:34:55.380928 1948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4ddbaaa5d8475d9384c87fd424b399" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318" is_pre_election: true
I20260809 05:34:55.381294 1948 raft_consensus.cc:2454] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 0.
I20260809 05:34:55.381888 1639 leader_election.cc:302] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:55.382236 2319 raft_consensus.cc:2781] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:55.382364 2319 raft_consensus.cc:482] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:55.382481 2319 raft_consensus.cc:3037] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:55.386116 2319 raft_consensus.cc:504] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.387476 1948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4ddbaaa5d8475d9384c87fd424b399" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318"
I20260809 05:34:55.387740 1948 raft_consensus.cc:3037] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:55.387996 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:55.374960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:59651 (local address 127.30.137.193:33255)
0809 05:34:55.375222 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:55.375230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:55.375715 (+ 485us) server_negotiation.cc:407] Connection header received
0809 05:34:55.376122 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:55.376137 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:55.376292 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:55.376394 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:55.377477 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.379003 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.383591 (+ 4588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.385784 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.386002 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:55.386127 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:55.387311 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0809 05:34:55.387532 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:34:55.389403 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:55.374869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33255 (local address 127.30.137.195:59651)
0809 05:34:55.375545 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:55.375562 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:55.375919 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:55.376733 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:55.376740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:55.376779 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:55.377245 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:55.377261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.379567 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.379576 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:55.383221 (+ 3645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:55.383250 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.386672 (+ 3422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.386682 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:55.386983 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:55.387073 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 05:34:55.388866 (+ 1793us) client_negotiation.cc:240] Negotiation successful
0809 05:34:55.389034 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":417,"threads_started":1}
I20260809 05:34:55.390712 2319 leader_election.cc:288] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 election: Requested vote from peers a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969), 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591)
I20260809 05:34:55.390553 2148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4ddbaaa5d8475d9384c87fd424b399" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" is_pre_election: true
I20260809 05:34:55.390942 2148 raft_consensus.cc:2454] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 0.
I20260809 05:34:55.391175 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4ddbaaa5d8475d9384c87fd424b399" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4728f2a6869e4b028db52b6b1e511aa7" is_pre_election: true
I20260809 05:34:55.391827 1744 raft_consensus.cc:2379] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in current term 1: Already voted for candidate 4728f2a6869e4b028db52b6b1e511aa7 in this term.
I20260809 05:34:55.392278 1948 raft_consensus.cc:2454] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 1.
I20260809 05:34:55.392827 1639 leader_election.cc:302] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:55.393146 2319 raft_consensus.cc:2781] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:55.393695 2319 raft_consensus.cc:686] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 LEADER]: Becoming Leader. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Running, Role: LEADER
I20260809 05:34:55.393750 2148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4ddbaaa5d8475d9384c87fd424b399" candidate_uuid: "4728f2a6869e4b028db52b6b1e511aa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2"
I20260809 05:34:55.393980 2148 raft_consensus.cc:3037] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:55.394064 2319 consensus_queue.cc:237] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:55.398267 2148 raft_consensus.cc:2454] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4728f2a6869e4b028db52b6b1e511aa7 in term 1.
I20260809 05:34:55.398736 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:55.375731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:39969 (local address 127.30.137.195:37485)
0809 05:34:55.389917 (+ 14186us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:55.389942 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:55.390075 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:55.390995 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:55.390999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:55.391030 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:55.391240 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:55.391255 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.392971 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.392978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:55.395419 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:55.395439 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.397793 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.397799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:55.398002 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:55.398040 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:34:55.398143 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 05:34:55.398314 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14062,"thread_start_us":14169,"threads_started":1}
I20260809 05:34:55.399524 2346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:55.375926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:37485 (local address 127.30.137.194:39969)
0809 05:34:55.389076 (+ 13150us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:55.389083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:55.390204 (+ 1121us) server_negotiation.cc:407] Connection header received
0809 05:34:55.390282 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:55.390286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:55.390369 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:55.390973 (+ 604us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:55.391464 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.392789 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.395707 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:55.397342 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:55.397509 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:55.397597 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:55.398980 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0809 05:34:55.399147 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12994,"thread_start_us":13539,"threads_started":1}
I20260809 05:34:55.400125 1948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4ddbaaa5d8475d9384c87fd424b399" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318" is_pre_election: true
I20260809 05:34:55.400508 1948 raft_consensus.cc:2379] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in current term 1: Already voted for candidate 4728f2a6869e4b028db52b6b1e511aa7 in this term.
I20260809 05:34:55.401115 2048 leader_election.cc:302] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95d1e6993b274ab0b7fab9b74c20b8d2; no voters: 4728f2a6869e4b028db52b6b1e511aa7, a804b80cfe104ef1bf253de5b5d86318
I20260809 05:34:55.401886 2322 raft_consensus.cc:2726] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 05:34:55.401587 1509 catalog_manager.cc:5495] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "4728f2a6869e4b028db52b6b1e511aa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } health_report { overall_health: UNKNOWN } } }
I20260809 05:34:55.491158 2148 raft_consensus.cc:1261] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Refusing update from remote peer 4728f2a6869e4b028db52b6b1e511aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:55.491683 1948 raft_consensus.cc:1261] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Refusing update from remote peer 4728f2a6869e4b028db52b6b1e511aa7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:55.491823 2267 consensus_queue.cc:1035] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:55.492332 2267 consensus_queue.cc:1035] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:34:55.519126 2302 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 05:34:57.110351 1505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:57.110605 1505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:57.115337 1505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Sending DeleteTablet for 3 replicas of tablet 5a4ddbaaa5d8475d9384c87fd424b399
I20260809 05:34:57.116236 1709 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4ddbaaa5d8475d9384c87fd424b399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:57 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260809 05:34:57.116427 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4ddbaaa5d8475d9384c87fd424b399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:57 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54782
I20260809 05:34:57.116464 2114 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4ddbaaa5d8475d9384c87fd424b399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:57 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:49914
I20260809 05:34:57.128793 2380 tablet_replica.cc:330] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: stopping tablet replica
I20260809 05:34:57.128885 2377 tablet_replica.cc:330] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: stopping tablet replica
I20260809 05:34:57.129251 2380 raft_consensus.cc:2229] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:57.129424 2377 raft_consensus.cc:2229] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:34:57.129657 2380 raft_consensus.cc:2258] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:57.130113 2377 raft_consensus.cc:2258] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:57.130388 2378 tablet_replica.cc:330] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: stopping tablet replica
I20260809 05:34:57.130715 2378 raft_consensus.cc:2229] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:57.131048 2378 raft_consensus.cc:2258] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:57.131063 2380 ts_tablet_manager.cc:1905] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:57.131234 2377 ts_tablet_manager.cc:1905] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:57.132390 2378 ts_tablet_manager.cc:1905] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:57.138628 2380 ts_tablet_manager.cc:1918] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 05:34:57.139010 2380 log.cc:1196] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/5a4ddbaaa5d8475d9384c87fd424b399
I20260809 05:34:57.139557 2380 ts_tablet_manager.cc:1939] T 5a4ddbaaa5d8475d9384c87fd424b399 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting consensus metadata
I20260809 05:34:57.140628 2378 ts_tablet_manager.cc:1918] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 05:34:57.140846 2378 log.cc:1196] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/5a4ddbaaa5d8475d9384c87fd424b399
I20260809 05:34:57.140941 1490 catalog_manager.cc:4841] TS 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591): tablet 5a4ddbaaa5d8475d9384c87fd424b399 (table test-table [id=55c43a0100d74ba29cc42e5fc71e5699]) successfully deleted
I20260809 05:34:57.141587 2378 ts_tablet_manager.cc:1939] T 5a4ddbaaa5d8475d9384c87fd424b399 P a804b80cfe104ef1bf253de5b5d86318: Deleting consensus metadata
I20260809 05:34:57.143038 1489 catalog_manager.cc:4841] TS a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969): tablet 5a4ddbaaa5d8475d9384c87fd424b399 (table test-table [id=55c43a0100d74ba29cc42e5fc71e5699]) successfully deleted
I20260809 05:34:57.147861 2377 ts_tablet_manager.cc:1918] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 05:34:57.148022 2377 log.cc:1196] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/5a4ddbaaa5d8475d9384c87fd424b399
I20260809 05:34:57.148494 2377 ts_tablet_manager.cc:1939] T 5a4ddbaaa5d8475d9384c87fd424b399 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting consensus metadata
I20260809 05:34:57.149731 1489 catalog_manager.cc:4841] TS 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255): tablet 5a4ddbaaa5d8475d9384c87fd424b399 (table test-table [id=55c43a0100d74ba29cc42e5fc71e5699]) successfully deleted
Aug 09 05:34:57 dist-test-slave-3kk6 krb5kdc[1441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 05:34:57 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:34:57.187155 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.171316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.30.137.254:42309)
0809 05:34:57.171980 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:57.171985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:57.172009 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:34:57.172177 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:57.172181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:57.172287 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:57.172481 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:57.173980 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.174910 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.176194 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.176634 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.176773 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.181074 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:57.181079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:57.181082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:57.181118 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:57.184365 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:57.185027 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:57.185039 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:57.185042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:57.185214 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:57.185589 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:57.185594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:57.185597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:57.185937 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:57.186160 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:57.186434 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 05:34:57.186622 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":354,"threads_started":1}
I20260809 05:34:57.190920 1505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:34:57.191605 1505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:57.199110 1709 tablet_service.cc:1467] Processing CreateTablet for tablet 7d09008e9533447c8e420f1f006ace2e (DEFAULT_TABLE table=test-table [id=fd74108870504388814e39d37f2e5e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:57.199079 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 7d09008e9533447c8e420f1f006ace2e (DEFAULT_TABLE table=test-table [id=fd74108870504388814e39d37f2e5e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:57.199390 2114 tablet_service.cc:1467] Processing CreateTablet for tablet 7d09008e9533447c8e420f1f006ace2e (DEFAULT_TABLE table=test-table [id=fd74108870504388814e39d37f2e5e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:57.199637 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d09008e9533447c8e420f1f006ace2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:57.199707 1709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d09008e9533447c8e420f1f006ace2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:57.199869 2114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d09008e9533447c8e420f1f006ace2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:57.205540 2391 tablet_bootstrap.cc:492] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Bootstrap starting.
I20260809 05:34:57.207424 2391 tablet_bootstrap.cc:654] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:57.207612 2392 tablet_bootstrap.cc:492] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Bootstrap starting.
I20260809 05:34:57.208608 2395 tablet_bootstrap.cc:492] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Bootstrap starting.
I20260809 05:34:57.210353 2392 tablet_bootstrap.cc:654] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:57.211155 2395 tablet_bootstrap.cc:654] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:57.213538 2392 tablet_bootstrap.cc:492] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: No bootstrap required, opened a new log
I20260809 05:34:57.213539 2391 tablet_bootstrap.cc:492] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: No bootstrap required, opened a new log
I20260809 05:34:57.213719 2392 ts_tablet_manager.cc:1397] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 05:34:57.213719 2391 ts_tablet_manager.cc:1397] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260809 05:34:57.214632 2392 raft_consensus.cc:348] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.214632 2391 raft_consensus.cc:348] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.214953 2391 raft_consensus.cc:374] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:57.214953 2392 raft_consensus.cc:374] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:57.215071 2391 raft_consensus.cc:729] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Initialized, Role: FOLLOWER
I20260809 05:34:57.215070 2392 raft_consensus.cc:729] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Initialized, Role: FOLLOWER
I20260809 05:34:57.215346 2392 consensus_queue.cc:260] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.215356 2391 consensus_queue.cc:260] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.215690 2395 tablet_bootstrap.cc:492] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: No bootstrap required, opened a new log
I20260809 05:34:57.215864 2395 ts_tablet_manager.cc:1397] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 05:34:57.216105 2391 ts_tablet_manager.cc:1428] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 05:34:57.217104 2392 ts_tablet_manager.cc:1428] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 05:34:57.217046 2395 raft_consensus.cc:348] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.217375 2395 raft_consensus.cc:374] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:57.217542 2395 raft_consensus.cc:729] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a804b80cfe104ef1bf253de5b5d86318, State: Initialized, Role: FOLLOWER
I20260809 05:34:57.217839 2395 consensus_queue.cc:260] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.219174 2395 ts_tablet_manager.cc:1428] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 05:34:57.490617 2405 raft_consensus.cc:482] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:57.490993 2405 raft_consensus.cc:504] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.492533 2405 leader_election.cc:288] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255), 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591)
I20260809 05:34:57.504484 2404 raft_consensus.cc:482] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:57.504799 2404 raft_consensus.cc:504] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.505879 2404 leader_election.cc:288] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:57.506577 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d09008e9533447c8e420f1f006ace2e" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4728f2a6869e4b028db52b6b1e511aa7" is_pre_election: true
I20260809 05:34:57.506601 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d09008e9533447c8e420f1f006ace2e" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318" is_pre_election: true
I20260809 05:34:57.506893 1949 raft_consensus.cc:2454] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 0.
I20260809 05:34:57.506922 1744 raft_consensus.cc:2454] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 0.
I20260809 05:34:57.507433 2048 leader_election.cc:302] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95d1e6993b274ab0b7fab9b74c20b8d2, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:57.507808 2404 raft_consensus.cc:2781] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:57.507963 2404 raft_consensus.cc:482] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:57.508069 2404 raft_consensus.cc:3037] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:57.511264 2404 raft_consensus.cc:504] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.511802 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.492945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33255 (local address 127.30.137.194:37763)
0809 05:34:57.503738 (+ 10793us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:57.503769 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:57.504040 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:57.505100 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:57.505108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:57.505141 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:57.505649 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.505662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.507376 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.507381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.509227 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.509242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.510902 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.510908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.511100 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.511137 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:34:57.511237 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 05:34:57.511396 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10591,"thread_start_us":10386,"threads_started":1}
I20260809 05:34:57.511824 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.493062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:37763 (local address 127.30.137.193:33255)
0809 05:34:57.504639 (+ 11577us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:57.504644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:57.504695 (+ 51us) server_negotiation.cc:407] Connection header received
0809 05:34:57.504778 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:57.504783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:57.504864 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:57.504974 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:57.505890 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.507206 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.509402 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.510715 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.510949 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.511057 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:57.511300 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 05:34:57.511445 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11395,"thread_start_us":11220,"threads_started":1}
I20260809 05:34:57.512365 2404 leader_election.cc:288] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 election: Requested vote from peers 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:57.512792 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d09008e9533447c8e420f1f006ace2e" candidate_uuid: "a804b80cfe104ef1bf253de5b5d86318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4728f2a6869e4b028db52b6b1e511aa7" is_pre_election: true
I20260809 05:34:57.512913 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d09008e9533447c8e420f1f006ace2e" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318"
I20260809 05:34:57.513101 1744 raft_consensus.cc:2454] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a804b80cfe104ef1bf253de5b5d86318 in term 0.
I20260809 05:34:57.513149 1949 raft_consensus.cc:3037] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:57.512986 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d09008e9533447c8e420f1f006ace2e" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4728f2a6869e4b028db52b6b1e511aa7"
I20260809 05:34:57.513391 1743 raft_consensus.cc:3037] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:57.513749 1845 leader_election.cc:302] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:57.513912 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.492955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:35591 (local address 127.30.137.194:59735)
0809 05:34:57.504233 (+ 11278us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:57.504260 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:57.504432 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:57.504948 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:57.504956 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:57.504982 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:34:57.505397 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.505409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.508830 (+ 3421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.508836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.510296 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.510308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.512593 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.512599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.513089 (+ 490us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.513141 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 05:34:57.513246 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 05:34:57.513414 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11112,"mutex_wait_us":25,"thread_start_us":10898,"threads_started":1}
I20260809 05:34:57.514636 2412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.493059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:59735 (local address 127.30.137.195:35591)
0809 05:34:57.503339 (+ 10280us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:57.503348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:57.504431 (+ 1083us) server_negotiation.cc:407] Connection header received
0809 05:34:57.504598 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:57.504605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:57.504697 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:57.504829 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:34:57.505621 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.508659 (+ 3038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.510532 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.512222 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.512405 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.512496 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:57.514135 (+ 1639us) server_negotiation.cc:299] Negotiation successful
0809 05:34:57.514289 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10103,"thread_start_us":9962,"threads_started":1}
I20260809 05:34:57.515295 2149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d09008e9533447c8e420f1f006ace2e" candidate_uuid: "a804b80cfe104ef1bf253de5b5d86318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" is_pre_election: true
I20260809 05:34:57.515626 2149 raft_consensus.cc:2379] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a804b80cfe104ef1bf253de5b5d86318 in current term 1: Already voted for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in this term.
I20260809 05:34:57.516407 1949 raft_consensus.cc:2454] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 1.
I20260809 05:34:57.516453 1743 raft_consensus.cc:2454] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 1.
I20260809 05:34:57.516647 2405 raft_consensus.cc:2741] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 05:34:57.516960 2048 leader_election.cc:302] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95d1e6993b274ab0b7fab9b74c20b8d2, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:57.517290 2404 raft_consensus.cc:2781] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:57.517817 2404 raft_consensus.cc:686] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 LEADER]: Becoming Leader. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Running, Role: LEADER
I20260809 05:34:57.518239 2404 consensus_queue.cc:237] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.521628 1505 catalog_manager.cc:5495] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } health_report { overall_health: HEALTHY } } }
I20260809 05:34:57.597940 2412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.592652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.30.137.195:35591)
0809 05:34:57.592838 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:57.592842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:57.592956 (+ 114us) server_negotiation.cc:407] Connection header received
0809 05:34:57.593162 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:57.593166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:57.593242 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:57.593387 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:57.593778 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.594537 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.595897 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.596328 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.596431 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.596855 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:57.597439 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:57.597544 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:57.597662 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 05:34:57.597728 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:34:57.613356 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.608140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:49789 (local address 127.30.137.193:33255)
0809 05:34:57.608388 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:57.608397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:57.608821 (+ 424us) server_negotiation.cc:407] Connection header received
0809 05:34:57.608978 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:57.608981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:57.609049 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:57.609124 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:57.609560 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.610477 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.611902 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.612308 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.612412 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.612787 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:57.612885 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:57.612949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:57.613076 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 05:34:57.613140 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:34:57.613469 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.608126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33255 (local address 127.30.137.195:49789)
0809 05:34:57.608710 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:57.608727 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:57.608860 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:57.609153 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:57.609156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:57.609166 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:57.609433 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.609441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.610590 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.610593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.611776 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.611789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.612470 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.612475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.612586 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.612643 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:57.612969 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:57.612978 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:34:57.613039 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 05:34:57.613143 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":324,"threads_started":1}
I20260809 05:34:57.622483 1949 raft_consensus.cc:1261] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Refusing update from remote peer 95d1e6993b274ab0b7fab9b74c20b8d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:57.622828 1743 raft_consensus.cc:1261] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Refusing update from remote peer 95d1e6993b274ab0b7fab9b74c20b8d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:57.623175 2413 consensus_queue.cc:1035] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:57.623636 2404 consensus_queue.cc:1035] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:57.655409 2412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.648900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:33923 (local address 127.30.137.195:35591)
0809 05:34:57.649054 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:57.649058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:57.649652 (+ 594us) server_negotiation.cc:407] Connection header received
0809 05:34:57.649934 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:57.649939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:57.650042 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:57.650143 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:34:57.650642 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.651765 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.653062 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.653643 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.653810 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.654318 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:34:57.654502 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:34:57.654608 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:57.654982 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 05:34:57.655067 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 05:34:57.655455 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:57.648861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:35591 (local address 127.30.137.193:33923)
0809 05:34:57.649545 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 05:34:57.649562 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:34:57.649749 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:34:57.650180 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:34:57.650185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:34:57.650197 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:34:57.650488 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.650502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.651896 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.651901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.652911 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:34:57.652922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:34:57.653885 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:34:57.653890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:34:57.654073 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:57.654123 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:34:57.654845 (+ 722us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:34:57.654855 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 05:34:57.654922 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 05:34:57.655029 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":393,"threads_started":1}
I20260809 05:34:57.696976 1509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:57.697290 1509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:57.700887 1509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Sending DeleteTablet for 3 replicas of tablet 7d09008e9533447c8e420f1f006ace2e
I20260809 05:34:57.702003 1709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d09008e9533447c8e420f1f006ace2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:57 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260809 05:34:57.702016 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d09008e9533447c8e420f1f006ace2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:57 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54782
I20260809 05:34:57.702546 2114 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d09008e9533447c8e420f1f006ace2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:57 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:49914
I20260809 05:34:57.702780 2436 tablet_replica.cc:330] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: stopping tablet replica
I20260809 05:34:57.703012 2436 raft_consensus.cc:2229] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:57.703590 2436 raft_consensus.cc:2258] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:57.703874 2440 tablet_replica.cc:330] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: stopping tablet replica
I20260809 05:34:57.704118 2440 raft_consensus.cc:2229] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:34:57.704298 1509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 05:34:57.704752 2440 raft_consensus.cc:2258] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:57.704890 2436 ts_tablet_manager.cc:1905] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 05:34:57.705302 1509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:34:57.705986 2440 ts_tablet_manager.cc:1905] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:57.712498 2436 ts_tablet_manager.cc:1918] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:57.712720 2436 log.cc:1196] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/7d09008e9533447c8e420f1f006ace2e
I20260809 05:34:57.713274 2436 ts_tablet_manager.cc:1939] T 7d09008e9533447c8e420f1f006ace2e P a804b80cfe104ef1bf253de5b5d86318: Deleting consensus metadata
I20260809 05:34:57.714131 2440 ts_tablet_manager.cc:1918] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:57.714309 2440 log.cc:1196] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/7d09008e9533447c8e420f1f006ace2e
I20260809 05:34:57.714547 1489 catalog_manager.cc:4841] TS a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969): tablet 7d09008e9533447c8e420f1f006ace2e (table test-table [id=fd74108870504388814e39d37f2e5e17]) successfully deleted
I20260809 05:34:57.715320 2440 ts_tablet_manager.cc:1939] T 7d09008e9533447c8e420f1f006ace2e P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting consensus metadata
I20260809 05:34:57.716480 1913 tablet_service.cc:1467] Processing CreateTablet for tablet 08789b4e2f284fe78a68028b324f8258 (DEFAULT_TABLE table=test-table [id=e1055fc22f524b28a9fd62273ac0ed4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:57.716435 1710 tablet_service.cc:1467] Processing CreateTablet for tablet 08789b4e2f284fe78a68028b324f8258 (DEFAULT_TABLE table=test-table [id=e1055fc22f524b28a9fd62273ac0ed4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:57.716946 1913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08789b4e2f284fe78a68028b324f8258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:57.716981 1710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08789b4e2f284fe78a68028b324f8258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:57.719074 2114 tablet_service.cc:1467] Processing CreateTablet for tablet 08789b4e2f284fe78a68028b324f8258 (DEFAULT_TABLE table=test-table [id=e1055fc22f524b28a9fd62273ac0ed4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:34:57.719496 2114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08789b4e2f284fe78a68028b324f8258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:34:57.726835 1490 catalog_manager.cc:4841] TS 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591): tablet 7d09008e9533447c8e420f1f006ace2e (table test-table [id=fd74108870504388814e39d37f2e5e17]) successfully deleted
I20260809 05:34:57.733661 2441 tablet_bootstrap.cc:492] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Bootstrap starting.
I20260809 05:34:57.733971 2445 tablet_bootstrap.cc:492] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Bootstrap starting.
I20260809 05:34:57.737087 2445 tablet_bootstrap.cc:654] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:57.737205 2441 tablet_bootstrap.cc:654] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:57.740154 2446 tablet_bootstrap.cc:492] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Bootstrap starting.
I20260809 05:34:57.744230 2446 tablet_bootstrap.cc:654] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Neither blocks nor log segments found. Creating new log.
I20260809 05:34:57.745752 2437 tablet_replica.cc:330] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: stopping tablet replica
I20260809 05:34:57.746207 2437 raft_consensus.cc:2229] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:57.746618 2437 raft_consensus.cc:2258] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:57.747473 2445 tablet_bootstrap.cc:492] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: No bootstrap required, opened a new log
I20260809 05:34:57.747715 2445 ts_tablet_manager.cc:1397] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260809 05:34:57.747961 2441 tablet_bootstrap.cc:492] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: No bootstrap required, opened a new log
I20260809 05:34:57.748245 2441 ts_tablet_manager.cc:1397] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260809 05:34:57.748587 2445 raft_consensus.cc:348] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.748767 2437 ts_tablet_manager.cc:1905] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:57.748956 2445 raft_consensus.cc:374] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:57.749115 2445 raft_consensus.cc:729] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a804b80cfe104ef1bf253de5b5d86318, State: Initialized, Role: FOLLOWER
I20260809 05:34:57.749118 2441 raft_consensus.cc:348] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.749505 2441 raft_consensus.cc:374] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:57.749467 2445 consensus_queue.cc:260] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.750788 2445 ts_tablet_manager.cc:1428] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 05:34:57.749660 2441 raft_consensus.cc:729] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4728f2a6869e4b028db52b6b1e511aa7, State: Initialized, Role: FOLLOWER
I20260809 05:34:57.752532 2441 consensus_queue.cc:260] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.755471 2441 ts_tablet_manager.cc:1428] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260809 05:34:57.757297 2437 ts_tablet_manager.cc:1918] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:34:57.757498 2437 log.cc:1196] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/7d09008e9533447c8e420f1f006ace2e
I20260809 05:34:57.758070 2437 ts_tablet_manager.cc:1939] T 7d09008e9533447c8e420f1f006ace2e P 4728f2a6869e4b028db52b6b1e511aa7: Deleting consensus metadata
I20260809 05:34:57.758898 2446 tablet_bootstrap.cc:492] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: No bootstrap required, opened a new log
I20260809 05:34:57.759140 2446 ts_tablet_manager.cc:1397] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.006s
I20260809 05:34:57.759297 1489 catalog_manager.cc:4841] TS 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255): tablet 7d09008e9533447c8e420f1f006ace2e (table test-table [id=fd74108870504388814e39d37f2e5e17]) successfully deleted
I20260809 05:34:57.759984 2446 raft_consensus.cc:348] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.760182 2446 raft_consensus.cc:374] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:34:57.760317 2446 raft_consensus.cc:729] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Initialized, Role: FOLLOWER
I20260809 05:34:57.760644 2446 consensus_queue.cc:260] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:57.761165 2446 ts_tablet_manager.cc:1428] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 05:34:58.136756 2428 raft_consensus.cc:482] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:34:58.136996 2428 raft_consensus.cc:504] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:58.137651 2428 leader_election.cc:288] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:58.138420 1948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08789b4e2f284fe78a68028b324f8258" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318" is_pre_election: true
I20260809 05:34:58.138418 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08789b4e2f284fe78a68028b324f8258" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4728f2a6869e4b028db52b6b1e511aa7" is_pre_election: true
I20260809 05:34:58.138702 1743 raft_consensus.cc:2454] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 0.
I20260809 05:34:58.138711 1948 raft_consensus.cc:2454] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 0.
I20260809 05:34:58.139181 2048 leader_election.cc:302] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4728f2a6869e4b028db52b6b1e511aa7, 95d1e6993b274ab0b7fab9b74c20b8d2; no voters:
I20260809 05:34:58.139485 2428 raft_consensus.cc:2781] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:34:58.139626 2428 raft_consensus.cc:482] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:34:58.139750 2428 raft_consensus.cc:3037] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:58.141997 2428 raft_consensus.cc:504] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:58.142571 2428 leader_election.cc:288] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 election: Requested vote from peers 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255), a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969)
I20260809 05:34:58.143152 1743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08789b4e2f284fe78a68028b324f8258" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4728f2a6869e4b028db52b6b1e511aa7"
I20260809 05:34:58.143250 1948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08789b4e2f284fe78a68028b324f8258" candidate_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a804b80cfe104ef1bf253de5b5d86318"
I20260809 05:34:58.143407 1743 raft_consensus.cc:3037] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:58.143462 1948 raft_consensus.cc:3037] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:34:58.146497 1948 raft_consensus.cc:2454] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 1.
I20260809 05:34:58.146553 1743 raft_consensus.cc:2454] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95d1e6993b274ab0b7fab9b74c20b8d2 in term 1.
I20260809 05:34:58.147074 2048 leader_election.cc:302] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95d1e6993b274ab0b7fab9b74c20b8d2, a804b80cfe104ef1bf253de5b5d86318; no voters:
I20260809 05:34:58.147396 2428 raft_consensus.cc:2781] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:34:58.147574 2428 raft_consensus.cc:686] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 LEADER]: Becoming Leader. State: Replica: 95d1e6993b274ab0b7fab9b74c20b8d2, State: Running, Role: LEADER
I20260809 05:34:58.147917 2428 consensus_queue.cc:237] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } }
I20260809 05:34:58.150748 1509 catalog_manager.cc:5495] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95d1e6993b274ab0b7fab9b74c20b8d2" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35591 } health_report { overall_health: HEALTHY } } }
I20260809 05:34:58.203475 1948 raft_consensus.cc:1261] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Refusing update from remote peer 95d1e6993b274ab0b7fab9b74c20b8d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:58.203522 1743 raft_consensus.cc:1261] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Refusing update from remote peer 95d1e6993b274ab0b7fab9b74c20b8d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:34:58.204078 2427 consensus_queue.cc:1035] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4728f2a6869e4b028db52b6b1e511aa7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:34:58.204456 2428 consensus_queue.cc:1035] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a804b80cfe104ef1bf253de5b5d86318" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 39969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:34:58.226586 2456 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 05:34:59.823887 1508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:59.824113 1508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:34:59.828153 1508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a2bb3b33cc0421c81d148d8ff53b9b6: Sending DeleteTablet for 3 replicas of tablet 08789b4e2f284fe78a68028b324f8258
I20260809 05:34:59.828959 1709 tablet_service.cc:1514] Processing DeleteTablet for tablet 08789b4e2f284fe78a68028b324f8258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:59 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260809 05:34:59.829067 2114 tablet_service.cc:1514] Processing DeleteTablet for tablet 08789b4e2f284fe78a68028b324f8258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:59 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:49914
I20260809 05:34:59.829134 1913 tablet_service.cc:1514] Processing DeleteTablet for tablet 08789b4e2f284fe78a68028b324f8258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:34:59 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54782
I20260809 05:34:59.829715 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1608
I20260809 05:34:59.834275 2496 tablet_replica.cc:330] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: stopping tablet replica
I20260809 05:34:59.834692 2496 raft_consensus.cc:2229] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:59.835151 2496 raft_consensus.cc:2258] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:59.836606 2496 ts_tablet_manager.cc:1905] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:59.846451 2502 tablet_replica.cc:330] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: stopping tablet replica
I20260809 05:34:59.846541 2496 ts_tablet_manager.cc:1918] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 05:34:59.846745 2497 tablet_replica.cc:330] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: stopping tablet replica
I20260809 05:34:59.846797 2496 log.cc:1196] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/08789b4e2f284fe78a68028b324f8258
I20260809 05:34:59.846777 2502 raft_consensus.cc:2229] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:34:59.847007 2497 raft_consensus.cc:2229] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:34:59.847083 2502 raft_consensus.cc:2258] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:59.847560 2497 raft_consensus.cc:2258] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:34:59.847905 2496 ts_tablet_manager.cc:1939] T 08789b4e2f284fe78a68028b324f8258 P 4728f2a6869e4b028db52b6b1e511aa7: Deleting consensus metadata
I20260809 05:34:59.848218 2502 ts_tablet_manager.cc:1905] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:59.848722 2497 ts_tablet_manager.cc:1905] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:34:59.849252 1489 catalog_manager.cc:4841] TS 4728f2a6869e4b028db52b6b1e511aa7 (127.30.137.193:33255): tablet 08789b4e2f284fe78a68028b324f8258 (table test-table [id=e1055fc22f524b28a9fd62273ac0ed4d]) successfully deleted
Aug 09 05:34:59 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:34:59.854275 2497 ts_tablet_manager.cc:1918] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 05:34:59.854441 2497 log.cc:1196] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/08789b4e2f284fe78a68028b324f8258
I20260809 05:34:59.855008 2497 ts_tablet_manager.cc:1939] T 08789b4e2f284fe78a68028b324f8258 P 95d1e6993b274ab0b7fab9b74c20b8d2: Deleting consensus metadata
I20260809 05:34:59.854979 2502 ts_tablet_manager.cc:1918] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 05:34:59.855134 2502 log.cc:1196] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/08789b4e2f284fe78a68028b324f8258
I20260809 05:34:59.855638 2502 ts_tablet_manager.cc:1939] T 08789b4e2f284fe78a68028b324f8258 P a804b80cfe104ef1bf253de5b5d86318: Deleting consensus metadata
I20260809 05:34:59.856115 1490 catalog_manager.cc:4841] TS 95d1e6993b274ab0b7fab9b74c20b8d2 (127.30.137.195:35591): tablet 08789b4e2f284fe78a68028b324f8258 (table test-table [id=e1055fc22f524b28a9fd62273ac0ed4d]) successfully deleted
I20260809 05:34:59.856753 1489 catalog_manager.cc:4841] TS a804b80cfe104ef1bf253de5b5d86318 (127.30.137.194:39969): tablet 08789b4e2f284fe78a68028b324f8258 (table test-table [id=e1055fc22f524b28a9fd62273ac0ed4d]) successfully deleted
I20260809 05:34:59.859863 2504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:59.831359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58106 (local address 127.30.137.193:33255)
0809 05:34:59.834577 (+ 3218us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:59.834586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:59.834619 (+ 33us) server_negotiation.cc:407] Connection header received
0809 05:34:59.834721 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:59.834736 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:59.834848 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:59.835000 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:59.837552 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:59.839313 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:59.841971 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:59.842989 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:59.843163 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:59.852470 (+ 9307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:59.852495 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:59.852516 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:59.852570 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:59.856788 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:59.857529 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:59.857543 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:59.857545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:59.857696 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:59.858098 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:59.858104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:59.858107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:59.858583 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:59.858965 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:59.859242 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 05:34:59.859491 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3016,"thread_start_us":2827,"threads_started":1}
W20260809 05:34:59.860523 1804 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58106
W20260809 05:34:59.861119 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1608: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:34:59.861315 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 1608
I20260809 05:34:59.902025 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1822
Aug 09 05:34:59 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:34:59.918803 2505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:59.902815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.30.137.194:39969)
0809 05:34:59.903387 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:59.903392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:59.903407 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:34:59.903464 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:59.903467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:59.903530 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:59.903677 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:59.905091 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:59.905856 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:59.907229 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:59.907677 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:59.907824 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:59.912356 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:59.912371 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:59.912386 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:59.912423 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:59.915571 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:59.916352 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:59.916360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:59.916362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:59.916550 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:59.916971 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:59.916975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:59.916977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:59.917375 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:59.917705 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:59.918059 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 05:34:59.918324 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":281,"threads_started":1}
W20260809 05:34:59.919634 2012 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60332
W20260809 05:34:59.920315 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1822: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:34:59.920468 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 1822
I20260809 05:34:59.951109 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2025
Aug 09 05:34:59 dist-test-slave-3kk6 krb5kdc[1441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:34:59.967743 2506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:34:59.951946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.30.137.195:35591)
0809 05:34:59.952656 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 05:34:59.952661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:34:59.952679 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:34:59.952739 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:34:59.952744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:34:59.952806 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:34:59.952906 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:34:59.954263 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:59.955117 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:59.956329 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:34:59.956784 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:34:59.956885 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:34:59.961669 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:34:59.961689 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:34:59.961705 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:34:59.961748 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:34:59.964660 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:59.965447 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:59.965455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:59.965457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:59.965608 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:34:59.966042 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:34:59.966047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:34:59.966053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:34:59.966431 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:34:59.966807 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0809 05:34:59.967120 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 05:34:59.967375 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":386,"threads_started":1}
W20260809 05:34:59.968526 2219 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54156
W20260809 05:34:59.969210 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2025: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:34:59.969381 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 2025
I20260809 05:35:00.005065 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 1468
I20260809 05:35:00.006276 1561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:00.129642 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 1468
2026-08-09T05:35:00Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9732 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:00 dist-test-slave-3kk6 krb5kdc[2513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:00 dist-test-slave-3kk6 krb5kdc[2513](info): set up 2 sockets
Aug 09 05:35:00 dist-test-slave-3kk6 krb5kdc[2513](info): commencing operation
krb5kdc: starting...
W20260809 05:35:01.456489 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:01 dist-test-slave-3kk6 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:01.691286 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42797
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:42797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:02.038254 2529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:02.038600 2529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:02.038779 2529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:02.046325 2529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:02.046465 2529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:02.046515 2529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:02.046568 2529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:02.046615 2529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:02.057106 2529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:42797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:02.058506 2529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:02.059850 2529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:02.072537 2537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:02.072571 2546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:02.072644 2538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:02.073239 2529 server_base.cc:1034] running on GCE node
I20260809 05:35:02.074206 2529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:02.075866 2529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:02.077121 2529 hybrid_clock.cc:648] HybridClock initialized: now 1786253702077090 us; error 53 us; skew 500 ppm
I20260809 05:35:02.077502 2529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:02 dist-test-slave-3kk6 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253702, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:02.081027 2529 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:02.082721 2529 webserver.cc:460] Webserver started at http://127.30.137.254:33065/ using document root <none> and password file <none>
I20260809 05:35:02.083475 2529 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:02.083631 2529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:02.083988 2529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:02.087366 2529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "0eee69371ab04ba2bb5f875e96f64422"
format_stamp: "Formatted at 2026-08-09 05:35:02 on dist-test-slave-3kk6"
server_key: "ff037ad383e68051cfd5b18e9ab63b81"
server_key_iv: "c6e81ff2be354c4143a6cce3b60c6120"
server_key_version: "encryptionkey@0"
I20260809 05:35:02.088245 2529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "0eee69371ab04ba2bb5f875e96f64422"
format_stamp: "Formatted at 2026-08-09 05:35:02 on dist-test-slave-3kk6"
server_key: "ff037ad383e68051cfd5b18e9ab63b81"
server_key_iv: "c6e81ff2be354c4143a6cce3b60c6120"
server_key_version: "encryptionkey@0"
I20260809 05:35:02.093879 2529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:02.098881 2554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:02.100682 2529 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260809 05:35:02.100911 2529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "0eee69371ab04ba2bb5f875e96f64422"
format_stamp: "Formatted at 2026-08-09 05:35:02 on dist-test-slave-3kk6"
server_key: "ff037ad383e68051cfd5b18e9ab63b81"
server_key_iv: "c6e81ff2be354c4143a6cce3b60c6120"
server_key_version: "encryptionkey@0"
I20260809 05:35:02.101186 2529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:02.126997 2529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:02.132263 2529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:02.132603 2529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:02.153437 2529 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:42797
I20260809 05:35:02.153469 2630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:42797 every 8 connection(s)
I20260809 05:35:02.155099 2529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:02.156312 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 2529
I20260809 05:35:02.156590 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:02.157423 31271 external_mini_cluster.cc:1428] Setting key d52950f9a9ccaa7be5ff9ba4b09c11ab
I20260809 05:35:02.160437 2632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:35:02 dist-test-slave-3kk6 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:02.175072 2632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422: Bootstrap starting.
I20260809 05:35:02.178535 2632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:02.179968 2632 log.cc:824] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:02.183192 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:02.159706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.30.137.254:42797)
0809 05:35:02.160665 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:02.160694 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:02.160795 (+ 101us) server_negotiation.cc:407] Connection header received
0809 05:35:02.162547 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:02.162570 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:02.163066 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:02.163435 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:02.165457 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:02.166974 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:02.168754 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:02.169468 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:02.169619 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:02.175044 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:02.175057 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:02.175074 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:02.175116 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:02.178406 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:02.179476 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:02.179484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:02.179491 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:02.179689 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:02.180171 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:02.180175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:02.180177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:02.180962 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:02.181314 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:02.181665 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 05:35:02.182086 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":420,"threads_started":1}
I20260809 05:35:02.183440 2632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422: No bootstrap required, opened a new log
I20260809 05:35:02.191228 2632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eee69371ab04ba2bb5f875e96f64422" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42797 } }
I20260809 05:35:02.191653 2632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:02.191780 2632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eee69371ab04ba2bb5f875e96f64422, State: Initialized, Role: FOLLOWER
I20260809 05:35:02.192613 2632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eee69371ab04ba2bb5f875e96f64422" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42797 } }
I20260809 05:35:02.192927 2632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:02.193074 2632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:02.193274 2632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:02.195694 2632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eee69371ab04ba2bb5f875e96f64422" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42797 } }
I20260809 05:35:02.196288 2632 leader_election.cc:302] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0eee69371ab04ba2bb5f875e96f64422; no voters:
I20260809 05:35:02.196919 2632 leader_election.cc:288] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:02.197080 2640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:02.197783 2640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [term 1 LEADER]: Becoming Leader. State: Replica: 0eee69371ab04ba2bb5f875e96f64422, State: Running, Role: LEADER
I20260809 05:35:02.198381 2640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eee69371ab04ba2bb5f875e96f64422" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42797 } }
I20260809 05:35:02.199136 2632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:02.201795 2642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eee69371ab04ba2bb5f875e96f64422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eee69371ab04ba2bb5f875e96f64422" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42797 } } }
I20260809 05:35:02.202152 2642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:02.202997 2643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eee69371ab04ba2bb5f875e96f64422. Latest consensus state: current_term: 1 leader_uuid: "0eee69371ab04ba2bb5f875e96f64422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0eee69371ab04ba2bb5f875e96f64422" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42797 } } }
I20260809 05:35:02.203390 2643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:02.204682 2653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:02.210407 2653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:02.220234 2653 catalog_manager.cc:1347] Generated new cluster ID: 2ca3eafe4fff468eaa8371722f87603c
I20260809 05:35:02.220384 2653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:02.238317 2653 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:02.239815 2653 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:02.251870 2653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eee69371ab04ba2bb5f875e96f64422: Generated new TSK 0
I20260809 05:35:02.252874 2653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:35:02.322764 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42797
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:02.706723 2674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:02.707103 2674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:02.707279 2674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:02.715379 2674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:02.715587 2674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:02.715781 2674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:02.726285 2674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:02.727803 2674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:02.729311 2674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:02.743722 2680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:02.743813 2681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:02.743767 2683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:02.750109 2674 server_base.cc:1034] running on GCE node
I20260809 05:35:02.751041 2674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:02.752208 2674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:02.753484 2674 hybrid_clock.cc:648] HybridClock initialized: now 1786253702753429 us; error 55 us; skew 500 ppm
I20260809 05:35:02.753850 2674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:02 dist-test-slave-3kk6 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253702, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:02.757404 2674 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:35:02.759421 2674 webserver.cc:460] Webserver started at http://127.30.137.193:33393/ using document root <none> and password file <none>
I20260809 05:35:02.760160 2674 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:02.760269 2674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:02.760556 2674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:02.763774 2674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "7636ae0adc9a4c798ab1005cf0e76a83"
format_stamp: "Formatted at 2026-08-09 05:35:02 on dist-test-slave-3kk6"
server_key: "8a3f828dc9e207c965f495087ccc09a7"
server_key_iv: "b5121e7e4f4a7ee95ab0f86f006512fd"
server_key_version: "encryptionkey@0"
I20260809 05:35:02.764603 2674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "7636ae0adc9a4c798ab1005cf0e76a83"
format_stamp: "Formatted at 2026-08-09 05:35:02 on dist-test-slave-3kk6"
server_key: "8a3f828dc9e207c965f495087ccc09a7"
server_key_iv: "b5121e7e4f4a7ee95ab0f86f006512fd"
server_key_version: "encryptionkey@0"
I20260809 05:35:02.770147 2674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:02.773896 2692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:02.775382 2674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:02.775563 2674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "7636ae0adc9a4c798ab1005cf0e76a83"
format_stamp: "Formatted at 2026-08-09 05:35:02 on dist-test-slave-3kk6"
server_key: "8a3f828dc9e207c965f495087ccc09a7"
server_key_iv: "b5121e7e4f4a7ee95ab0f86f006512fd"
server_key_version: "encryptionkey@0"
I20260809 05:35:02.775786 2674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:02.796428 2674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:02.801422 2674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:02.801774 2674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:02.802883 2674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:02.804534 2674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:02.804621 2674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:02.804803 2674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:02.804909 2674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:02.840848 2674 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:45007
I20260809 05:35:02.840898 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:45007 every 8 connection(s)
I20260809 05:35:02.842217 2674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:02.852411 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 2674
I20260809 05:35:02.852653 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:02.853437 31271 external_mini_cluster.cc:1428] Setting key a015a8a7e3c82de34fdebf2256e6238d
Aug 09 05:35:02 dist-test-slave-3kk6 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253702, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:02.864828 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:02.844950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:43543 (local address 127.30.137.254:42797)
0809 05:35:02.845472 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:02.845479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:02.846463 (+ 984us) server_negotiation.cc:407] Connection header received
0809 05:35:02.848406 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:02.848411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:02.848496 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:02.848600 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:02.850147 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:02.850965 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:02.853062 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:02.853465 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:02.853625 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:02.858200 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:02.858205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:02.858209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:02.858242 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:02.860993 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:02.862003 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:02.862007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:02.862010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:02.862135 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:02.862694 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:02.862704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:02.862708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:02.862980 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:02.863096 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:02.864081 (+ 985us) server_negotiation.cc:299] Negotiation successful
0809 05:35:02.864349 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260809 05:35:02.865512 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:02.845543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42797 (local address 127.30.137.193:43543)
0809 05:35:02.846269 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:02.846324 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:02.848126 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:02.848824 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:02.848832 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:02.849397 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:02.849974 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:02.849992 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:02.851125 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:02.851129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:02.852907 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:02.852925 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:02.853746 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:02.853754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:02.854041 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:02.854950 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:02.854988 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:02.857933 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:02.861156 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:02.861169 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:02.861193 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:02.861880 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:02.862250 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:02.862257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:02.862263 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:02.862536 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:02.863120 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:02.863131 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:02.863640 (+ 509us) client_negotiation.cc:769] Sending connection context
0809 05:35:02.863936 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 05:35:02.864359 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":286,"threads_started":1}
I20260809 05:35:02.867216 2844 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42797
I20260809 05:35:02.867573 2844 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:02.868574 2844 heartbeater.cc:511] Master 127.30.137.254:42797 requested a full tablet report, sending...
I20260809 05:35:02.871186 2581 ts_manager.cc:194] Registered new tserver with Master: 7636ae0adc9a4c798ab1005cf0e76a83 (127.30.137.193:45007)
I20260809 05:35:02.873047 2581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:43543
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:35:02.910528 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42797
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:03.269552 2851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:03.269939 2851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:03.270109 2851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:03.277786 2851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:03.277985 2851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:03.278174 2851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:03.288982 2851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:03.290458 2851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:03.292287 2851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:03.307016 2858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:03.307418 2861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:03.307029 2857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:03.308005 2851 server_base.cc:1034] running on GCE node
I20260809 05:35:03.308725 2851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:03.309798 2851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:03.311045 2851 hybrid_clock.cc:648] HybridClock initialized: now 1786253703310994 us; error 49 us; skew 500 ppm
I20260809 05:35:03.311393 2851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:03 dist-test-slave-3kk6 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253703, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:03.314853 2851 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:35:03.316587 2851 webserver.cc:460] Webserver started at http://127.30.137.194:35193/ using document root <none> and password file <none>
I20260809 05:35:03.317323 2851 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:03.317436 2851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:03.317716 2851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:03.320933 2851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "00c086038b484b19b09c36451148f0c7"
format_stamp: "Formatted at 2026-08-09 05:35:03 on dist-test-slave-3kk6"
server_key: "2e625108c161b3de821fca0b3f3b6d9a"
server_key_iv: "ae4b03b72b07393d7aa854f3e91ee2fa"
server_key_version: "encryptionkey@0"
I20260809 05:35:03.321784 2851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "00c086038b484b19b09c36451148f0c7"
format_stamp: "Formatted at 2026-08-09 05:35:03 on dist-test-slave-3kk6"
server_key: "2e625108c161b3de821fca0b3f3b6d9a"
server_key_iv: "ae4b03b72b07393d7aa854f3e91ee2fa"
server_key_version: "encryptionkey@0"
I20260809 05:35:03.327829 2851 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260809 05:35:03.331707 2870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:03.333178 2851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:03.333333 2851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "00c086038b484b19b09c36451148f0c7"
format_stamp: "Formatted at 2026-08-09 05:35:03 on dist-test-slave-3kk6"
server_key: "2e625108c161b3de821fca0b3f3b6d9a"
server_key_iv: "ae4b03b72b07393d7aa854f3e91ee2fa"
server_key_version: "encryptionkey@0"
I20260809 05:35:03.333541 2851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:03.357892 2851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:03.362495 2851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:03.362872 2851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:03.364298 2851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:03.365877 2851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:03.365955 2851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:03.366075 2851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:03.366139 2851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:03.402908 2851 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:44169
I20260809 05:35:03.402961 3045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:44169 every 8 connection(s)
I20260809 05:35:03.404295 2851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:03.406373 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 2851
I20260809 05:35:03.406594 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:03.407099 31271 external_mini_cluster.cc:1428] Setting key 04487b22eb4b99f4a835e021151147b0
Aug 09 05:35:03 dist-test-slave-3kk6 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253703, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:03.426642 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:03.407436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:51217 (local address 127.30.137.254:42797)
0809 05:35:03.408002 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:03.408009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:03.409455 (+ 1446us) server_negotiation.cc:407] Connection header received
0809 05:35:03.411612 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:03.411618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:03.411752 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:03.411896 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:03.413447 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.414193 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.415844 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.416320 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.416461 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:03.420555 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:03.420561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:03.420564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:03.420597 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:03.423432 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:03.424197 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:03.424201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:03.424204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:03.424337 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:03.424797 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:03.424802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:03.424804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:03.425087 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:03.425250 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:03.425989 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 05:35:03.426234 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":284,"threads_started":1}
I20260809 05:35:03.427680 3052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:03.408072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42797 (local address 127.30.137.194:51217)
0809 05:35:03.408851 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:03.408894 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:03.411331 (+ 2437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:03.412147 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:03.412157 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:03.412695 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:03.413271 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:03.413288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.414367 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.414376 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:03.415692 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:03.415706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.416574 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.416582 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:03.416807 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:03.417655 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:03.417689 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:03.420295 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:03.423587 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:03.423596 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:03.423614 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:03.424055 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:03.424457 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:03.424463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:03.424467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:03.424664 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:03.425228 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:03.425236 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:03.425600 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 05:35:03.425881 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 05:35:03.426459 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":335,"threads_started":1}
I20260809 05:35:03.429232 3049 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42797
I20260809 05:35:03.429564 3049 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
I20260809 05:35:03.430333 3049 heartbeater.cc:511] Master 127.30.137.254:42797 requested a full tablet report, sending...
I20260809 05:35:03.432094 2581 ts_manager.cc:194] Registered new tserver with Master: 00c086038b484b19b09c36451148f0c7 (127.30.137.194:44169)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
I20260809 05:35:03.433238 2581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:51217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:35:03.461534 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42797
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:32841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:03.813596 3056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:03.813921 3056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:03.814111 3056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:03.821545 3056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:03.821697 3056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:03.821905 3056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:03.832208 3056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:32841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:03.833606 3056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:03.835227 3056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:03.849789 3067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:03.849759 3069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:03.849858 3066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:03.855903 3056 server_base.cc:1034] running on GCE node
I20260809 05:35:03.856635 3056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:03.857687 3056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:03.858932 3056 hybrid_clock.cc:648] HybridClock initialized: now 1786253703858885 us; error 44 us; skew 500 ppm
I20260809 05:35:03.859277 3056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:03 dist-test-slave-3kk6 krb5kdc[2513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253703, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:03.862686 3056 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:35:03.864295 3056 webserver.cc:460] Webserver started at http://127.30.137.195:36823/ using document root <none> and password file <none>
I20260809 05:35:03.865082 3056 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:03.865247 3056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:03.865592 3056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:03.868762 3056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "0f01426812544c77ab08937912323ef4"
format_stamp: "Formatted at 2026-08-09 05:35:03 on dist-test-slave-3kk6"
server_key: "50adef5164f4c163e5e855ccebabf22e"
server_key_iv: "8dbb63365bd129052c8c9ac3e3bfc5c8"
server_key_version: "encryptionkey@0"
I20260809 05:35:03.869630 3056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "0f01426812544c77ab08937912323ef4"
format_stamp: "Formatted at 2026-08-09 05:35:03 on dist-test-slave-3kk6"
server_key: "50adef5164f4c163e5e855ccebabf22e"
server_key_iv: "8dbb63365bd129052c8c9ac3e3bfc5c8"
server_key_version: "encryptionkey@0"
I20260809 05:35:03.875164 3056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:35:03.877274 2844 heartbeater.cc:503] Master 127.30.137.254:42797 was elected leader, sending a full tablet report...
I20260809 05:35:03.879133 3085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:03.880735 3056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:35:03.880929 3056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "0f01426812544c77ab08937912323ef4"
format_stamp: "Formatted at 2026-08-09 05:35:03 on dist-test-slave-3kk6"
server_key: "50adef5164f4c163e5e855ccebabf22e"
server_key_iv: "8dbb63365bd129052c8c9ac3e3bfc5c8"
server_key_version: "encryptionkey@0"
I20260809 05:35:03.881175 3056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:03.919358 3056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:03.923820 3056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:03.924170 3056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:03.925267 3056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:03.926785 3056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:03.926900 3056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:03.927063 3056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:03.927139 3056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:03.962885 3056 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:45971
I20260809 05:35:03.963004 3251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:45971 every 8 connection(s)
I20260809 05:35:03.964351 3056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:03.966845 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 3056
I20260809 05:35:03.967053 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:03.967563 31271 external_mini_cluster.cc:1428] Setting key 7a87c57b4edeeb49cfc27fe6c181d804
Aug 09 05:35:03 dist-test-slave-3kk6 krb5kdc[2513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253703, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:03.985774 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:03.967276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:41925 (local address 127.30.137.254:42797)
0809 05:35:03.967957 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:03.967964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:03.968987 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 05:35:03.970956 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:03.970960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:03.971032 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:03.971122 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:03.972702 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.973503 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.975166 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.975572 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.975670 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:03.979305 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:03.979310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:03.979314 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:03.979350 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:03.982295 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:03.983353 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:03.983359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:03.983362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:03.983514 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:03.983936 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:03.983940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:03.983942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:03.984227 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:03.984351 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:03.985154 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 05:35:03.985367 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":275,"threads_started":1}
I20260809 05:35:03.986811 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:03.967908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42797 (local address 127.30.137.195:41925)
0809 05:35:03.968749 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:03.968807 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:03.970625 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:03.971355 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:03.971363 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:03.971937 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:03.972541 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:03.972560 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.973641 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.973645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:03.975026 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:03.975040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:03.975768 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:03.975772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:03.975966 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:03.976651 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:03.976674 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:03.979077 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:03.982456 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:03.982465 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:03.982481 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:03.983202 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:03.983618 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:03.983621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:03.983623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:03.983823 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:03.984346 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:03.984352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:03.984764 (+ 412us) client_negotiation.cc:769] Sending connection context
0809 05:35:03.985008 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 05:35:03.985506 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":371,"threads_started":1}
I20260809 05:35:03.988240 3254 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42797
I20260809 05:35:03.988586 3254 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:03.989355 3254 heartbeater.cc:511] Master 127.30.137.254:42797 requested a full tablet report, sending...
I20260809 05:35:03.990967 2581 ts_manager.cc:194] Registered new tserver with Master: 0f01426812544c77ab08937912323ef4 (127.30.137.195:45971)
I20260809 05:35:03.991858 2581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:41925
I20260809 05:35:03.995786 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 05:35:04.036799 3258 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:04.033914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.30.137.254:42797)
0809 05:35:04.034122 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:04.034127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:04.034701 (+ 574us) server_negotiation.cc:407] Connection header received
0809 05:35:04.034971 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:04.034976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:04.035063 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:04.035168 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:04.036473 (+ 1305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48260: BlockingRecv error: recv got EOF from 127.0.0.1:48260 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:35:04.040076 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2674
W20260809 05:35:04.043326 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2674: Not authorized: Client connection negotiation failed: client connection to 127.30.137.193:45007: server requires authentication, but client does not have Kerberos credentials available
I20260809 05:35:04.043529 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 2674
I20260809 05:35:04.068260 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2851
W20260809 05:35:04.071389 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2851: Not authorized: Client connection negotiation failed: client connection to 127.30.137.194:44169: server requires authentication, but client does not have Kerberos credentials available
I20260809 05:35:04.071589 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 2851
I20260809 05:35:04.092639 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3056
W20260809 05:35:04.096441 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3056: Not authorized: Client connection negotiation failed: client connection to 127.30.137.195:45971: server requires authentication, but client does not have Kerberos credentials available
I20260809 05:35:04.096612 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 3056
I20260809 05:35:04.121140 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 2529
I20260809 05:35:04.122541 2625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:04.230119 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 2529
2026-08-09T05:35:04Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4095 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:04 dist-test-slave-3kk6 krb5kdc[3287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:04 dist-test-slave-3kk6 krb5kdc[3287](info): set up 2 sockets
Aug 09 05:35:04 dist-test-slave-3kk6 krb5kdc[3287](info): commencing operation
krb5kdc: starting...
W20260809 05:35:05.539755 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:05 dist-test-slave-3kk6 krb5kdc[3287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:05.786439 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44949
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:44949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:06.153698 3312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:06.154036 3312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:06.154229 3312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:06.161888 3312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:06.162037 3312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:06.162153 3312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:06.162256 3312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:06.162350 3312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:06.172936 3312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:44949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:06.174404 3312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:06.175858 3312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:06.188521 3321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:06.188601 3322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:06.188568 3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:06.189401 3312 server_base.cc:1034] running on GCE node
I20260809 05:35:06.190210 3312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:06.191779 3312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:06.192987 3312 hybrid_clock.cc:648] HybridClock initialized: now 1786253706192942 us; error 44 us; skew 500 ppm
I20260809 05:35:06.193318 3312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:06 dist-test-slave-3kk6 krb5kdc[3287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253706, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:06.196720 3312 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:06.198431 3312 webserver.cc:460] Webserver started at http://127.30.137.254:39661/ using document root <none> and password file <none>
I20260809 05:35:06.199270 3312 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:06.199425 3312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:06.199784 3312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:06.203181 3312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "d69316cd1380420d90fea7e8962df144"
format_stamp: "Formatted at 2026-08-09 05:35:06 on dist-test-slave-3kk6"
server_key: "216c61093aba1afb8021b0fe1ad734b4"
server_key_iv: "1133d09862cf4e6132996fefea961c93"
server_key_version: "encryptionkey@0"
I20260809 05:35:06.204133 3312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "d69316cd1380420d90fea7e8962df144"
format_stamp: "Formatted at 2026-08-09 05:35:06 on dist-test-slave-3kk6"
server_key: "216c61093aba1afb8021b0fe1ad734b4"
server_key_iv: "1133d09862cf4e6132996fefea961c93"
server_key_version: "encryptionkey@0"
I20260809 05:35:06.210037 3312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:06.213987 3331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:06.215636 3312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:35:06.215854 3312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "d69316cd1380420d90fea7e8962df144"
format_stamp: "Formatted at 2026-08-09 05:35:06 on dist-test-slave-3kk6"
server_key: "216c61093aba1afb8021b0fe1ad734b4"
server_key_iv: "1133d09862cf4e6132996fefea961c93"
server_key_version: "encryptionkey@0"
I20260809 05:35:06.216084 3312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:06.241416 3312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:06.245774 3312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:06.246107 3312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:06.266117 3312 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:44949
I20260809 05:35:06.266153 3404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:44949 every 8 connection(s)
I20260809 05:35:06.267695 3312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:06.271733 3405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:06.273926 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 3312
I20260809 05:35:06.274183 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:06.275064 31271 external_mini_cluster.cc:1428] Setting key 0b464b23109030d1aa0b9ad430fd1e9e
I20260809 05:35:06.285511 3405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144: Bootstrap starting.
I20260809 05:35:06.289866 3405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144: Neither blocks nor log segments found. Creating new log.
Aug 09 05:35:06 dist-test-slave-3kk6 krb5kdc[3287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:06.291237 3405 log.cc:824] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:06.295223 3405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144: No bootstrap required, opened a new log
I20260809 05:35:06.302271 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:06.277146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.30.137.254:44949)
0809 05:35:06.278043 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:06.278059 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:06.278105 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:35:06.279964 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:06.279990 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:06.280423 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:06.280781 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:06.282861 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:06.284413 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:06.286296 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:06.287074 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:06.287248 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:06.292609 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:06.292617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:06.292634 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:06.292676 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:06.296904 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:06.297821 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:06.297833 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:06.297847 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:06.298094 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:06.298509 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:06.298514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:06.298517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:06.299375 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:06.299699 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:06.300188 (+ 489us) server_negotiation.cc:299] Negotiation successful
0809 05:35:06.300706 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":424,"threads_started":1}
I20260809 05:35:06.306393 3405 raft_consensus.cc:348] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69316cd1380420d90fea7e8962df144" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44949 } }
I20260809 05:35:06.306918 3405 raft_consensus.cc:374] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:06.307057 3405 raft_consensus.cc:729] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d69316cd1380420d90fea7e8962df144, State: Initialized, Role: FOLLOWER
I20260809 05:35:06.307186 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:35:06.307734 3405 consensus_queue.cc:260] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69316cd1380420d90fea7e8962df144" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44949 } }
I20260809 05:35:06.308017 3405 raft_consensus.cc:388] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:06.308157 3405 raft_consensus.cc:482] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:06.308328 3405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:06.312032 3405 raft_consensus.cc:504] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69316cd1380420d90fea7e8962df144" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44949 } }
I20260809 05:35:06.312678 3405 leader_election.cc:302] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d69316cd1380420d90fea7e8962df144; no voters:
I20260809 05:35:06.313421 3405 leader_election.cc:288] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:06.313578 3413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:06.314556 3413 raft_consensus.cc:686] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [term 1 LEADER]: Becoming Leader. State: Replica: d69316cd1380420d90fea7e8962df144, State: Running, Role: LEADER
I20260809 05:35:06.315189 3413 consensus_queue.cc:237] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69316cd1380420d90fea7e8962df144" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44949 } }
I20260809 05:35:06.316156 3405 sys_catalog.cc:564] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:06.318460 3414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d69316cd1380420d90fea7e8962df144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69316cd1380420d90fea7e8962df144" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44949 } } }
I20260809 05:35:06.318922 3414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:06.319633 3415 sys_catalog.cc:455] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d69316cd1380420d90fea7e8962df144. Latest consensus state: current_term: 1 leader_uuid: "d69316cd1380420d90fea7e8962df144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d69316cd1380420d90fea7e8962df144" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44949 } } }
I20260809 05:35:06.320024 3415 sys_catalog.cc:458] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:06.320199 3422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:06.333727 3422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:06.348222 3422 catalog_manager.cc:1347] Generated new cluster ID: f970046faf0240b38c902daff89e526b
I20260809 05:35:06.348469 3422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:06.361786 3422 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:06.363909 3422 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:06.368824 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:06.360813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.30.137.254:44949)
0809 05:35:06.361016 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:06.361026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:06.361636 (+ 610us) server_negotiation.cc:407] Connection header received
0809 05:35:06.361934 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:06.361940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:06.362022 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:06.362150 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:06.363318 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:06.364119 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:06.365430 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:06.365909 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:06.366047 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:06.367312 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:06.367317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:06.367320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:06.367978 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:06.368050 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:06.368076 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:06.368203 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:06.368343 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 05:35:06.368485 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:35:06.378618 3422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d69316cd1380420d90fea7e8962df144: Generated new TSK 0
I20260809 05:35:06.379791 3422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:35:06.390456 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3312
I20260809 05:35:06.391816 3398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:06.483722 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 3312
2026-08-09T05:35:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2248 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:06 dist-test-slave-3kk6 krb5kdc[3462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:06 dist-test-slave-3kk6 krb5kdc[3462](info): set up 2 sockets
Aug 09 05:35:06 dist-test-slave-3kk6 krb5kdc[3462](info): commencing operation
krb5kdc: starting...
W20260809 05:35:07.839450 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:07 dist-test-slave-3kk6 krb5kdc[3462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:08.086630 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39987
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:39987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:08.444736 3480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:08.445075 3480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:08.445240 3480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:08.452574 3480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:08.452689 3480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:08.452800 3480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:08.452915 3480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:08.452999 3480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:08.463886 3480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:39987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:08.465330 3480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:08.466709 3480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:08.479067 3492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:08.479245 3490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:08.479880 3489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:08.480607 3480 server_base.cc:1034] running on GCE node
I20260809 05:35:08.481519 3480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:08.483242 3480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:08.484495 3480 hybrid_clock.cc:648] HybridClock initialized: now 1786253708484455 us; error 43 us; skew 500 ppm
I20260809 05:35:08.484880 3480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:08 dist-test-slave-3kk6 krb5kdc[3462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253708, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:08.488382 3480 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:08.490196 3480 webserver.cc:460] Webserver started at http://127.30.137.254:33573/ using document root <none> and password file <none>
I20260809 05:35:08.491226 3480 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:08.491405 3480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:08.491781 3480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:08.495208 3480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "e3868b5e6b024cb090e54c2fd95d788e"
format_stamp: "Formatted at 2026-08-09 05:35:08 on dist-test-slave-3kk6"
server_key: "d33d6336c560fc025a67f7885c2748d8"
server_key_iv: "5e3cc6c34c4be994ce4d3dc993669bfe"
server_key_version: "encryptionkey@0"
I20260809 05:35:08.496088 3480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "e3868b5e6b024cb090e54c2fd95d788e"
format_stamp: "Formatted at 2026-08-09 05:35:08 on dist-test-slave-3kk6"
server_key: "d33d6336c560fc025a67f7885c2748d8"
server_key_iv: "5e3cc6c34c4be994ce4d3dc993669bfe"
server_key_version: "encryptionkey@0"
I20260809 05:35:08.501926 3480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 05:35:08.505823 3500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:08.507453 3480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:08.507680 3480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "e3868b5e6b024cb090e54c2fd95d788e"
format_stamp: "Formatted at 2026-08-09 05:35:08 on dist-test-slave-3kk6"
server_key: "d33d6336c560fc025a67f7885c2748d8"
server_key_iv: "5e3cc6c34c4be994ce4d3dc993669bfe"
server_key_version: "encryptionkey@0"
I20260809 05:35:08.507921 3480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:08.533046 3480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:08.537791 3480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:08.538153 3480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:08.558801 3480 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:39987
I20260809 05:35:08.558799 3572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:39987 every 8 connection(s)
I20260809 05:35:08.560377 3480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:08.563647 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 3480
I20260809 05:35:08.563907 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:08.564796 31271 external_mini_cluster.cc:1428] Setting key f917491cef4ad628704ddda2760d62f2
I20260809 05:35:08.564831 3574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:08.580228 3574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e: Bootstrap starting.
Aug 09 05:35:08 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:08.584574 3574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:08.586071 3574 log.cc:824] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:08.590003 3574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e: No bootstrap required, opened a new log
I20260809 05:35:08.591239 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:08.567056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.30.137.254:39987)
0809 05:35:08.567903 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:08.567914 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:08.567959 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:35:08.569843 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:08.569866 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:08.570257 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:08.570598 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:08.572483 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:08.573828 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:08.575642 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:08.576396 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:08.576539 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:08.581655 (+ 5116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:08.581675 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:08.581696 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:08.581766 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:08.586046 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:08.587173 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:08.587199 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:08.587211 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:08.587419 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:08.587888 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:08.587895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:08.587898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:08.588606 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:08.588931 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:08.589302 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 05:35:08.589780 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":382,"threads_started":1}
I20260809 05:35:08.599117 3574 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3868b5e6b024cb090e54c2fd95d788e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39987 } }
I20260809 05:35:08.599519 3574 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:08.599601 3574 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3868b5e6b024cb090e54c2fd95d788e, State: Initialized, Role: FOLLOWER
I20260809 05:35:08.600211 3574 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3868b5e6b024cb090e54c2fd95d788e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39987 } }
I20260809 05:35:08.600584 3574 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:08.600762 3574 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:08.600963 3574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:08.603391 3574 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3868b5e6b024cb090e54c2fd95d788e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39987 } }
I20260809 05:35:08.603924 3574 leader_election.cc:302] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3868b5e6b024cb090e54c2fd95d788e; no voters:
I20260809 05:35:08.604655 3574 leader_election.cc:288] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:08.604792 3583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:08.605768 3583 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [term 1 LEADER]: Becoming Leader. State: Replica: e3868b5e6b024cb090e54c2fd95d788e, State: Running, Role: LEADER
I20260809 05:35:08.606328 3583 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3868b5e6b024cb090e54c2fd95d788e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39987 } }
I20260809 05:35:08.606979 3574 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:08.609347 3586 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3868b5e6b024cb090e54c2fd95d788e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3868b5e6b024cb090e54c2fd95d788e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39987 } } }
I20260809 05:35:08.609683 3586 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:08.610599 3592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:08.610625 3587 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3868b5e6b024cb090e54c2fd95d788e. Latest consensus state: current_term: 1 leader_uuid: "e3868b5e6b024cb090e54c2fd95d788e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3868b5e6b024cb090e54c2fd95d788e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39987 } } }
I20260809 05:35:08.616470 3587 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:08.617344 3592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:08.628436 3592 catalog_manager.cc:1347] Generated new cluster ID: bb548dd4eae7489693fe2e242c7db03b
I20260809 05:35:08.628580 3592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:08.639437 3592 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:08.641374 3592 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:08.663157 3592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3868b5e6b024cb090e54c2fd95d788e: Generated new TSK 0
I20260809 05:35:08.664338 3592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:35:08.735695 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39987
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:09.096843 3615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:09.097241 3615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:09.097414 3615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:09.105206 3615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:09.105420 3615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:09.105605 3615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:09.116643 3615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:09.118085 3615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:09.119536 3615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:09.133948 3624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:09.133912 3625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:09.133929 3630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:09.140290 3615 server_base.cc:1034] running on GCE node
I20260809 05:35:09.141150 3615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:09.142349 3615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:09.143628 3615 hybrid_clock.cc:648] HybridClock initialized: now 1786253709143545 us; error 87 us; skew 500 ppm
I20260809 05:35:09.144022 3615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:09 dist-test-slave-3kk6 krb5kdc[3462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253709, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:09.147765 3615 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:35:09.149529 3615 webserver.cc:460] Webserver started at http://127.30.137.193:40487/ using document root <none> and password file <none>
I20260809 05:35:09.150300 3615 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:09.150470 3615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:09.150878 3615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:09.153997 3615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "f153c01e8d3f43ad92416b1280c4ae95"
format_stamp: "Formatted at 2026-08-09 05:35:09 on dist-test-slave-3kk6"
server_key: "a5ceacfd5dffeb94f35b11e99e1a9be1"
server_key_iv: "48d65dab040cdb5a6804d697c6b5fa17"
server_key_version: "encryptionkey@0"
I20260809 05:35:09.154978 3615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "f153c01e8d3f43ad92416b1280c4ae95"
format_stamp: "Formatted at 2026-08-09 05:35:09 on dist-test-slave-3kk6"
server_key: "a5ceacfd5dffeb94f35b11e99e1a9be1"
server_key_iv: "48d65dab040cdb5a6804d697c6b5fa17"
server_key_version: "encryptionkey@0"
I20260809 05:35:09.160485 3615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:35:09.164407 3642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:09.165885 3615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:35:09.166087 3615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "f153c01e8d3f43ad92416b1280c4ae95"
format_stamp: "Formatted at 2026-08-09 05:35:09 on dist-test-slave-3kk6"
server_key: "a5ceacfd5dffeb94f35b11e99e1a9be1"
server_key_iv: "48d65dab040cdb5a6804d697c6b5fa17"
server_key_version: "encryptionkey@0"
I20260809 05:35:09.166317 3615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:09.198097 3615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:09.202906 3615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:09.203260 3615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:09.204337 3615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:09.205892 3615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:09.206002 3615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:09.206156 3615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:09.206240 3615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:09.242547 3615 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:34565
I20260809 05:35:09.242575 3809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:34565 every 8 connection(s)
I20260809 05:35:09.243963 3615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:09.253042 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 3615
I20260809 05:35:09.253298 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:09.253981 31271 external_mini_cluster.cc:1428] Setting key 8fe486d777d5c1bed9713bc3b430b1cb
Aug 09 05:35:09 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253709, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:09.266641 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:09.246878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:49725 (local address 127.30.137.254:39987)
0809 05:35:09.247370 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:09.247374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:09.248462 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 05:35:09.250576 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:09.250582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:09.250749 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:09.250888 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:09.252612 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.253535 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.255598 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.256057 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.256219 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:09.260230 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:09.260237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:09.260241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:09.260280 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:09.263310 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:09.264165 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:09.264170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:09.264173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:09.264306 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:09.264789 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:09.264793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:09.264795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:09.265077 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:09.265240 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:09.265998 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 05:35:09.266248 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":253,"threads_started":1}
I20260809 05:35:09.267699 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:09.247504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39987 (local address 127.30.137.193:49725)
0809 05:35:09.248258 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:09.248309 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:09.250269 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:09.251153 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:09.251163 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:09.251766 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:09.252413 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:09.252434 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.253729 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.253734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:09.255424 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:09.255439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.256337 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.256346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:09.256603 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:09.257383 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:09.257409 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:09.259941 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:09.263519 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:09.263528 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:09.263545 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:09.264012 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:09.264428 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:09.264433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:09.264437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:09.264650 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:09.265213 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:09.265230 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:09.265598 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 05:35:09.265875 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 05:35:09.266486 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":300,"threads_started":1}
I20260809 05:35:09.269428 3810 heartbeater.cc:347] Connected to a master server at 127.30.137.254:39987
I20260809 05:35:09.269784 3810 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:09.270738 3810 heartbeater.cc:511] Master 127.30.137.254:39987 requested a full tablet report, sending...
I20260809 05:35:09.273250 3522 ts_manager.cc:194] Registered new tserver with Master: f153c01e8d3f43ad92416b1280c4ae95 (127.30.137.193:34565)
I20260809 05:35:09.275681 3522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:49725
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:35:09.309608 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39987
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:09.661911 3816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:09.662252 3816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:09.662422 3816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:09.670331 3816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:09.670472 3816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:09.670679 3816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:09.681139 3816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:09.682592 3816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:09.684067 3816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:09.698220 3828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:09.698333 3832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:09.698266 3836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:09.699438 3816 server_base.cc:1034] running on GCE node
I20260809 05:35:09.700109 3816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:09.701149 3816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:09.702404 3816 hybrid_clock.cc:648] HybridClock initialized: now 1786253709702357 us; error 41 us; skew 500 ppm
I20260809 05:35:09.702821 3816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:09 dist-test-slave-3kk6 krb5kdc[3462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253709, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:09.706281 3816 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:35:09.708033 3816 webserver.cc:460] Webserver started at http://127.30.137.194:42055/ using document root <none> and password file <none>
I20260809 05:35:09.708751 3816 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:09.708904 3816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:09.709229 3816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:09.712618 3816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "e7f3ba68aefe495fba7814f199ca6dbf"
format_stamp: "Formatted at 2026-08-09 05:35:09 on dist-test-slave-3kk6"
server_key: "357bbf909d98c1f24f0445d32848456e"
server_key_iv: "df625024872fb99e7dfd9a0af1c77e18"
server_key_version: "encryptionkey@0"
I20260809 05:35:09.713619 3816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "e7f3ba68aefe495fba7814f199ca6dbf"
format_stamp: "Formatted at 2026-08-09 05:35:09 on dist-test-slave-3kk6"
server_key: "357bbf909d98c1f24f0445d32848456e"
server_key_iv: "df625024872fb99e7dfd9a0af1c77e18"
server_key_version: "encryptionkey@0"
I20260809 05:35:09.719094 3816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 05:35:09.722848 3847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:09.724242 3816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:09.724431 3816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "e7f3ba68aefe495fba7814f199ca6dbf"
format_stamp: "Formatted at 2026-08-09 05:35:09 on dist-test-slave-3kk6"
server_key: "357bbf909d98c1f24f0445d32848456e"
server_key_iv: "df625024872fb99e7dfd9a0af1c77e18"
server_key_version: "encryptionkey@0"
I20260809 05:35:09.724646 3816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:09.736339 3816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:09.741052 3816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:09.741410 3816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:09.742441 3816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:09.743920 3816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:09.744029 3816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:09.744176 3816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:09.744256 3816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:09.780377 3816 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46137
I20260809 05:35:09.780421 4019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46137 every 8 connection(s)
I20260809 05:35:09.781842 3816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:09.785589 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 3816
I20260809 05:35:09.785830 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:09.786468 31271 external_mini_cluster.cc:1428] Setting key 1f5195bab7b2ebd8652e6ff902626f44
Aug 09 05:35:09 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253709, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:09.805104 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:09.784816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:33235 (local address 127.30.137.254:39987)
0809 05:35:09.785301 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:09.785306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:09.786635 (+ 1329us) server_negotiation.cc:407] Connection header received
0809 05:35:09.788854 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:09.788859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:09.788955 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:09.789081 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:09.790844 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.791592 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.793518 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.793961 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.794107 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:09.798275 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:09.798280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:09.798284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:09.798321 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:09.801217 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:09.802411 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:09.802415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:09.802417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:09.802551 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:09.803133 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:09.803137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:09.803139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:09.803423 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:09.803539 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:09.804613 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0809 05:35:09.804809 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":263,"threads_started":1}
I20260809 05:35:09.806957 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:09.785418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39987 (local address 127.30.137.194:33235)
0809 05:35:09.786368 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:09.786411 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:09.788558 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:09.789353 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:09.789363 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:09.789936 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:09.790577 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:09.790599 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.791803 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.791814 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:09.793329 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:09.793352 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:09.794231 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:09.794240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:09.794494 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:09.795380 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:09.795404 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:09.797965 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:09.801420 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:09.801434 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:09.801457 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:09.802264 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:09.802699 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:09.802707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:09.802713 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:09.803003 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:09.803574 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:09.803587 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:09.804172 (+ 585us) client_negotiation.cc:769] Sending connection context
0809 05:35:09.804516 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 05:35:09.805223 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":457,"threads_started":1}
I20260809 05:35:09.808696 4021 heartbeater.cc:347] Connected to a master server at 127.30.137.254:39987
I20260809 05:35:09.809099 4021 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:09.810128 4021 heartbeater.cc:511] Master 127.30.137.254:39987 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
I20260809 05:35:09.812029 3522 ts_manager.cc:194] Registered new tserver with Master: e7f3ba68aefe495fba7814f199ca6dbf (127.30.137.194:46137)
I20260809 05:35:09.813156 3522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:33235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:35:09.846271 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39987
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:10.214072 4030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:10.214407 4030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:10.214609 4030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:10.223011 4030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:10.223218 4030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:10.223448 4030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:10.238555 4030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:39987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:10.246825 4030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:10.249040 4030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:10.268117 4040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:10.268148 4046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:10.268224 4044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:10.274966 4030 server_base.cc:1034] running on GCE node
I20260809 05:35:10.276001 4030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:10.277585 4030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:10.278858 4030 hybrid_clock.cc:648] HybridClock initialized: now 1786253710278794 us; error 63 us; skew 500 ppm
I20260809 05:35:10.279382 4030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:10.279379 3810 heartbeater.cc:503] Master 127.30.137.254:39987 was elected leader, sending a full tablet report...
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:10.284498 4030 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:35:10.286268 4030 webserver.cc:460] Webserver started at http://127.30.137.195:37951/ using document root <none> and password file <none>
I20260809 05:35:10.287175 4030 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:10.287343 4030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:10.287700 4030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:10.291119 4030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "be96a70252154f5081ee02caf1e41aa4"
format_stamp: "Formatted at 2026-08-09 05:35:10 on dist-test-slave-3kk6"
server_key: "d5de874bde53ca98e7eed74975757429"
server_key_iv: "4c2acc0ef104c5156e4035d470931041"
server_key_version: "encryptionkey@0"
I20260809 05:35:10.292073 4030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "be96a70252154f5081ee02caf1e41aa4"
format_stamp: "Formatted at 2026-08-09 05:35:10 on dist-test-slave-3kk6"
server_key: "d5de874bde53ca98e7eed74975757429"
server_key_iv: "4c2acc0ef104c5156e4035d470931041"
server_key_version: "encryptionkey@0"
I20260809 05:35:10.298185 4030 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 05:35:10.302177 4055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:10.303911 4030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:10.304131 4030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "be96a70252154f5081ee02caf1e41aa4"
format_stamp: "Formatted at 2026-08-09 05:35:10 on dist-test-slave-3kk6"
server_key: "d5de874bde53ca98e7eed74975757429"
server_key_iv: "4c2acc0ef104c5156e4035d470931041"
server_key_version: "encryptionkey@0"
I20260809 05:35:10.304387 4030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:10.318346 4030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:10.323158 4030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:10.323519 4030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:10.324625 4030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:10.326125 4030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:10.326241 4030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:10.326414 4030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:10.326498 4030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:10.363736 4030 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:34905
I20260809 05:35:10.363801 4248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:34905 every 8 connection(s)
I20260809 05:35:10.365201 4030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:10.375190 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 4030
I20260809 05:35:10.375404 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:10.375896 31271 external_mini_cluster.cc:1428] Setting key fff4ad61f479e0b2cdc4fd635f5f5e03
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:10.386765 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:10.368157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:37527 (local address 127.30.137.254:39987)
0809 05:35:10.368637 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:10.368642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:10.369765 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 05:35:10.371817 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:10.371823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:10.371929 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:10.372058 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:10.373663 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.374470 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.376451 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.377026 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.377168 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:10.381090 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:10.381095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:10.381099 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:10.381134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:10.383768 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.384580 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.384584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.384587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.384706 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.385103 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.385106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.385108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.385378 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:10.385490 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:10.386228 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 05:35:10.386409 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
I20260809 05:35:10.387810 4261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:10.368784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39987 (local address 127.30.137.195:37527)
0809 05:35:10.369583 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:10.369623 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:10.371503 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:10.372317 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:10.372326 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:10.372879 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:10.373472 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:10.373491 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.374705 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.374712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:10.376281 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:10.376296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.377279 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.377287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:10.377540 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:10.378276 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:10.378300 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:10.380867 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:10.383918 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:10.383926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:10.383941 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:10.384458 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:10.384799 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:10.384802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:10.384805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:10.384996 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:10.385496 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:10.385502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:10.385866 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 05:35:10.386116 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 05:35:10.386555 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":319,"threads_started":1}
I20260809 05:35:10.389317 4252 heartbeater.cc:347] Connected to a master server at 127.30.137.254:39987
I20260809 05:35:10.389714 4252 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:10.390798 4252 heartbeater.cc:511] Master 127.30.137.254:39987 requested a full tablet report, sending...
I20260809 05:35:10.392508 3522 ts_manager.cc:194] Registered new tserver with Master: be96a70252154f5081ee02caf1e41aa4 (127.30.137.195:34905)
I20260809 05:35:10.393334 3522 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:37527
I20260809 05:35:10.404760 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:10.465600 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:10.450565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.30.137.254:39987)
0809 05:35:10.450881 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:10.450890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:10.451298 (+ 408us) server_negotiation.cc:407] Connection header received
0809 05:35:10.451545 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:10.451550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:10.451622 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:10.451745 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:10.453212 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.453989 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.455359 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.455785 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.455936 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:10.460202 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:10.460208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:10.460211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:10.460253 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:10.462958 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.463746 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.463751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.463753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.463912 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.464305 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.464308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.464310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.464660 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:10.464867 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:10.465149 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 05:35:10.465346 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 05:35:10.466284 3522 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38146
I20260809 05:35:10.468699 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3615
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:10.487025 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:10.469887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.30.137.193:34565)
0809 05:35:10.470539 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:10.470546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:10.470570 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:35:10.470722 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:10.470735 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:10.471018 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:10.471205 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:10.472514 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.473624 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.475001 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.475618 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.475778 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:10.480505 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:10.480523 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:10.480539 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:10.480591 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:10.483788 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.484620 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.484630 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.484632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.484789 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.485210 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.485216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.485221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.485625 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:10.485994 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:10.486282 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 05:35:10.486580 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":330,"threads_started":1}
W20260809 05:35:10.488462 3803 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59618
W20260809 05:35:10.489704 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3615: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:35:10.489951 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 3615
I20260809 05:35:10.517040 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3816
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:35:10.535053 4277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:10.518119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.30.137.194:46137)
0809 05:35:10.518770 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:10.518787 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:10.518821 (+ 34us) server_negotiation.cc:407] Connection header received
0809 05:35:10.518891 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:10.518897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:10.519130 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:10.519342 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:10.520825 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.521890 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.523312 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.523918 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.524095 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:10.528856 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:10.528871 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:10.528886 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:10.528928 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:10.532004 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.532824 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.532832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.532835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.532979 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.533369 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.533374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.533376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.533807 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:10.534089 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:10.534356 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 05:35:10.534594 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":263,"threads_started":1}
W20260809 05:35:10.536473 4013 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53664
W20260809 05:35:10.537729 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3816: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:35:10.537945 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 3816
I20260809 05:35:10.563571 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 4030
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[3462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253710, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:35:10.582442 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:10.564766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.30.137.195:34905)
0809 05:35:10.565602 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:10.565642 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:10.565687 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:35:10.565788 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:10.565800 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:10.566036 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:10.566239 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:10.567843 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.568907 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.570280 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:10.571076 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:10.571232 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:10.576177 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:10.576196 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:10.576213 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:10.576256 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:10.579287 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.580135 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.580144 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.580147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.580282 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:10.580694 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:10.580700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:10.580705 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:10.581099 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:10.581451 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:10.581760 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 05:35:10.582047 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":437,"threads_started":1}
W20260809 05:35:10.583807 4241 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52488
W20260809 05:35:10.585003 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 4030: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:35:10.585206 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 4030
I20260809 05:35:10.609253 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 3480
I20260809 05:35:10.610590 3567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:10.717935 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 3480
2026-08-09T05:35:10Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4238 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[4283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[4283](info): set up 2 sockets
Aug 09 05:35:10 dist-test-slave-3kk6 krb5kdc[4283](info): commencing operation
krb5kdc: starting...
W20260809 05:35:12.027253 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:12 dist-test-slave-3kk6 krb5kdc[4283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:12.274885 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:33373
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:33373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:12.623003 4303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:12.623320 4303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:12.623486 4303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:12.631028 4303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:12.631146 4303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:12.631248 4303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:12.631348 4303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:12.631438 4303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:12.641645 4303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:33373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:33373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:12.643149 4303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:12.644536 4303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:12.656908 4311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:12.656932 4316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:12.657006 4310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:12.658212 4303 server_base.cc:1034] running on GCE node
I20260809 05:35:12.659076 4303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:12.660630 4303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:12.661864 4303 hybrid_clock.cc:648] HybridClock initialized: now 1786253712661819 us; error 43 us; skew 500 ppm
I20260809 05:35:12.662190 4303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:12 dist-test-slave-3kk6 krb5kdc[4283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253712, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:12.670132 4303 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:12.671932 4303 webserver.cc:460] Webserver started at http://127.30.137.254:45981/ using document root <none> and password file <none>
I20260809 05:35:12.672631 4303 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:12.672741 4303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:12.673019 4303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:12.676192 4303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "98cb1ba3b87d439ca2291233ba5bfce3"
format_stamp: "Formatted at 2026-08-09 05:35:12 on dist-test-slave-3kk6"
server_key: "2476e518c1d51f1741c405013ce1c1d3"
server_key_iv: "4f9b6025f762c18c2361099dfa94c41b"
server_key_version: "encryptionkey@0"
I20260809 05:35:12.676987 4303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "98cb1ba3b87d439ca2291233ba5bfce3"
format_stamp: "Formatted at 2026-08-09 05:35:12 on dist-test-slave-3kk6"
server_key: "2476e518c1d51f1741c405013ce1c1d3"
server_key_iv: "4f9b6025f762c18c2361099dfa94c41b"
server_key_version: "encryptionkey@0"
I20260809 05:35:12.682772 4303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:35:12.686970 4327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:12.688515 4303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:12.688722 4303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "98cb1ba3b87d439ca2291233ba5bfce3"
format_stamp: "Formatted at 2026-08-09 05:35:12 on dist-test-slave-3kk6"
server_key: "2476e518c1d51f1741c405013ce1c1d3"
server_key_iv: "4f9b6025f762c18c2361099dfa94c41b"
server_key_version: "encryptionkey@0"
I20260809 05:35:12.688973 4303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:12.710206 4303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:12.715166 4303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:12.715497 4303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:12.735085 4303 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:33373
I20260809 05:35:12.735121 4407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:33373 every 8 connection(s)
I20260809 05:35:12.736531 4303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:12.738953 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 4303
I20260809 05:35:12.739217 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:12.740080 31271 external_mini_cluster.cc:1428] Setting key 0e5ccf32ebff353d6bee2f2b16cbebf9
I20260809 05:35:12.741285 4408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:35:12 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:12.756636 4408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3: Bootstrap starting.
I20260809 05:35:12.760671 4408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:12.761791 4408 log.cc:824] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:12.765208 4408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3: No bootstrap required, opened a new log
I20260809 05:35:12.765422 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:12.742235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.30.137.254:33373)
0809 05:35:12.743086 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:12.743098 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:12.743139 (+ 41us) server_negotiation.cc:407] Connection header received
0809 05:35:12.744768 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:12.744789 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:12.745175 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:12.745521 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:12.747564 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:12.749016 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:12.750809 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:12.751533 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:12.751682 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:12.756686 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:12.756695 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:12.756735 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:12.756797 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:12.760713 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:12.761505 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:12.761513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:12.761520 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:12.761704 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:12.762195 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:12.762199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:12.762201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:12.763114 (+ 913us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:12.763406 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:12.763908 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 05:35:12.764349 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":421,"threads_started":1}
I20260809 05:35:12.772969 4408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33373 } }
I20260809 05:35:12.773398 4408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:12.773519 4408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98cb1ba3b87d439ca2291233ba5bfce3, State: Initialized, Role: FOLLOWER
I20260809 05:35:12.774080 4408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33373 } }
I20260809 05:35:12.774333 4408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:12.774482 4408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:12.774727 4408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:12.777091 4408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33373 } }
I20260809 05:35:12.777606 4408 leader_election.cc:302] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98cb1ba3b87d439ca2291233ba5bfce3; no voters:
I20260809 05:35:12.778196 4408 leader_election.cc:288] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:12.778353 4416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:12.779479 4416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [term 1 LEADER]: Becoming Leader. State: Replica: 98cb1ba3b87d439ca2291233ba5bfce3, State: Running, Role: LEADER
I20260809 05:35:12.780097 4416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33373 } }
I20260809 05:35:12.780553 4408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:12.782989 4419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98cb1ba3b87d439ca2291233ba5bfce3. Latest consensus state: current_term: 1 leader_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33373 } } }
I20260809 05:35:12.783438 4419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:12.784034 4418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98cb1ba3b87d439ca2291233ba5bfce3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33373 } } }
I20260809 05:35:12.784345 4418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:12.784485 4430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:12.790539 4430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:12.800720 4430 catalog_manager.cc:1347] Generated new cluster ID: f1b9238e7a084556bcea66650684aaa3
I20260809 05:35:12.800860 4430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:12.821573 4430 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:12.823099 4430 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:12.836905 4430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98cb1ba3b87d439ca2291233ba5bfce3: Generated new TSK 0
I20260809 05:35:12.837795 4430 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:35:12.915179 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33373
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:13.295830 4446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:13.296164 4446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:13.296342 4446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:13.303761 4446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:13.303939 4446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:13.304121 4446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:13.314534 4446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:13.315876 4446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:13.317324 4446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:13.330785 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:13.330732 4453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:13.330819 4452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:13.331575 4446 server_base.cc:1034] running on GCE node
I20260809 05:35:13.332222 4446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:13.333189 4446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:13.334436 4446 hybrid_clock.cc:648] HybridClock initialized: now 1786253713334379 us; error 56 us; skew 500 ppm
I20260809 05:35:13.334847 4446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:13 dist-test-slave-3kk6 krb5kdc[4283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:13.338344 4446 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:35:13.339960 4446 webserver.cc:460] Webserver started at http://127.30.137.193:39137/ using document root <none> and password file <none>
I20260809 05:35:13.340698 4446 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:13.340854 4446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:13.341181 4446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:13.344241 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "570287831b09406f84b9e1c0b41f3ef3"
format_stamp: "Formatted at 2026-08-09 05:35:13 on dist-test-slave-3kk6"
server_key: "9ce5657b9bdaab7b299d9e428dea7735"
server_key_iv: "caa52f802c8f81a60ca4844f429b0bf2"
server_key_version: "encryptionkey@0"
I20260809 05:35:13.345085 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "570287831b09406f84b9e1c0b41f3ef3"
format_stamp: "Formatted at 2026-08-09 05:35:13 on dist-test-slave-3kk6"
server_key: "9ce5657b9bdaab7b299d9e428dea7735"
server_key_iv: "caa52f802c8f81a60ca4844f429b0bf2"
server_key_version: "encryptionkey@0"
I20260809 05:35:13.350420 4446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:35:13.353935 4469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:13.355511 4446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:13.355698 4446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "570287831b09406f84b9e1c0b41f3ef3"
format_stamp: "Formatted at 2026-08-09 05:35:13 on dist-test-slave-3kk6"
server_key: "9ce5657b9bdaab7b299d9e428dea7735"
server_key_iv: "caa52f802c8f81a60ca4844f429b0bf2"
server_key_version: "encryptionkey@0"
I20260809 05:35:13.355938 4446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:13.377815 4446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:13.382630 4446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:13.382983 4446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:13.384019 4446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:13.385481 4446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:13.385588 4446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:13.385738 4446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:13.385821 4446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:13.419755 4446 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:45993
I20260809 05:35:13.419842 4654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:45993 every 8 connection(s)
I20260809 05:35:13.421243 4446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:13.424350 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 4446
I20260809 05:35:13.424589 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:13.425489 31271 external_mini_cluster.cc:1428] Setting key b6cf4f51b1f0815103b7b468a7c05d1f
Aug 09 05:35:13 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:13.445891 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:13.424059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:45115 (local address 127.30.137.254:33373)
0809 05:35:13.424635 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:13.424643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:13.425771 (+ 1128us) server_negotiation.cc:407] Connection header received
0809 05:35:13.428147 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:13.428151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:13.428227 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:13.428309 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:13.430560 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.431322 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.433810 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.434166 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.434267 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:13.438736 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:13.438746 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:13.438754 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:13.438799 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:13.442509 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:13.443280 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:13.443288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:13.443294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:13.443451 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:13.443876 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:13.443883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:13.443890 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:13.444289 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:13.444447 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:13.445140 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 05:35:13.445415 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":285,"threads_started":1}
I20260809 05:35:13.446461 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:13.424637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33373 (local address 127.30.137.193:45115)
0809 05:35:13.425563 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:13.425635 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:13.427821 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:13.428630 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:13.428645 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:13.429507 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:13.430357 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:13.430387 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.431489 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.431497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:13.433654 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:13.433673 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.434404 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.434413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:13.434744 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:13.435531 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:13.435565 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:13.438475 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:13.442709 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:13.442721 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:13.442733 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:13.443152 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:13.443566 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:13.443569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:13.443572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:13.443750 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:13.444411 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:13.444417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:13.444723 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 05:35:13.444945 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 05:35:13.445343 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":313,"threads_started":1}
I20260809 05:35:13.448113 4656 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33373
I20260809 05:35:13.448437 4656 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:13.449220 4656 heartbeater.cc:511] Master 127.30.137.254:33373 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:35:13.451498 4353 ts_manager.cc:194] Registered new tserver with Master: 570287831b09406f84b9e1c0b41f3ef3 (127.30.137.193:45993)
I20260809 05:35:13.453162 4353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:45115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:35:13.484244 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33373
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:13.826076 4665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:13.826366 4665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:13.826462 4665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:13.833629 4665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:13.833745 4665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:13.833859 4665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:13.843361 4665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:13.844584 4665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:13.845877 4665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:13.859776 4672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:13.859702 4674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:13.859653 4677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:13.861133 4665 server_base.cc:1034] running on GCE node
I20260809 05:35:13.861779 4665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:13.862835 4665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:13.864038 4665 hybrid_clock.cc:648] HybridClock initialized: now 1786253713863993 us; error 45 us; skew 500 ppm
I20260809 05:35:13.864329 4665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:13 dist-test-slave-3kk6 krb5kdc[4283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:13.867619 4665 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:35:13.869182 4665 webserver.cc:460] Webserver started at http://127.30.137.194:44797/ using document root <none> and password file <none>
I20260809 05:35:13.869894 4665 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:13.870035 4665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:13.870332 4665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:13.873354 4665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "fd64913056b0433799678beade04388a"
format_stamp: "Formatted at 2026-08-09 05:35:13 on dist-test-slave-3kk6"
server_key: "07c01f627c7dfce9a29c0de90ef09510"
server_key_iv: "8c2a4537319d1d726ac520d9006e030d"
server_key_version: "encryptionkey@0"
I20260809 05:35:13.874119 4665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "fd64913056b0433799678beade04388a"
format_stamp: "Formatted at 2026-08-09 05:35:13 on dist-test-slave-3kk6"
server_key: "07c01f627c7dfce9a29c0de90ef09510"
server_key_iv: "8c2a4537319d1d726ac520d9006e030d"
server_key_version: "encryptionkey@0"
I20260809 05:35:13.879621 4665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 05:35:13.883420 4685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:13.884871 4665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:13.885052 4665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "fd64913056b0433799678beade04388a"
format_stamp: "Formatted at 2026-08-09 05:35:13 on dist-test-slave-3kk6"
server_key: "07c01f627c7dfce9a29c0de90ef09510"
server_key_iv: "8c2a4537319d1d726ac520d9006e030d"
server_key_version: "encryptionkey@0"
I20260809 05:35:13.885283 4665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:13.906256 4665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:13.910492 4665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:13.910816 4665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:13.911772 4665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:13.913089 4665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:13.913164 4665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:13.913260 4665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:13.913304 4665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:13.947954 4665 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:41125
I20260809 05:35:13.948009 4864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:41125 every 8 connection(s)
I20260809 05:35:13.949376 4665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:13.959606 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 4665
I20260809 05:35:13.959863 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:13.960382 31271 external_mini_cluster.cc:1428] Setting key 2dea35485657d6c388b627c324dabf3a
Aug 09 05:35:13 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:13.972268 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:13.952322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:32799 (local address 127.30.137.254:33373)
0809 05:35:13.952810 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:13.952818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:13.953828 (+ 1010us) server_negotiation.cc:407] Connection header received
0809 05:35:13.955713 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:13.955717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:13.955795 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:13.955901 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:13.957398 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.958106 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.960071 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.960705 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.960811 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:13.965608 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:13.965613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:13.965616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:13.965650 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:13.968407 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:13.969397 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:13.969402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:13.969404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:13.969540 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:13.970134 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:13.970142 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:13.970148 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:13.970534 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:13.970715 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:13.971602 (+ 887us) server_negotiation.cc:299] Negotiation successful
0809 05:35:13.971860 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":234,"threads_started":1}
I20260809 05:35:13.973235 4871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:13.952923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33373 (local address 127.30.137.194:32799)
0809 05:35:13.953635 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:13.953674 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:13.955448 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:13.956134 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:13.956142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:13.956663 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:13.957243 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:13.957260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.958235 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.958239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:13.959892 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:13.959909 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:13.960952 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:13.960962 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:13.961212 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:13.961996 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:13.962023 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:13.965371 (+ 3348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:13.968555 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:13.968567 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:13.968586 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:13.969239 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:13.969656 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:13.969663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:13.969691 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:13.969983 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:13.970709 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:13.970719 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:13.971143 (+ 424us) client_negotiation.cc:769] Sending connection context
0809 05:35:13.971499 (+ 356us) client_negotiation.cc:240] Negotiation successful
0809 05:35:13.972057 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":272,"threads_started":1}
I20260809 05:35:13.974772 4866 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33373
I20260809 05:35:13.975169 4866 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:13.976068 4866 heartbeater.cc:511] Master 127.30.137.254:33373 requested a full tablet report, sending...
I20260809 05:35:13.977818 4353 ts_manager.cc:194] Registered new tserver with Master: fd64913056b0433799678beade04388a (127.30.137.194:41125)
I20260809 05:35:13.979209 4353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:32799
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:35:14.029861 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33373
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:14.400023 4876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:14.400322 4876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:14.400475 4876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:14.407969 4876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:14.408085 4876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:14.408249 4876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:14.418118 4876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:14.419504 4876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:14.420794 4876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:14.434159 4888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:14.434234 4889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:14.434281 4894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:14.435405 4876 server_base.cc:1034] running on GCE node
I20260809 05:35:14.436090 4876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:14.437119 4876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:14.438354 4876 hybrid_clock.cc:648] HybridClock initialized: now 1786253714438310 us; error 62 us; skew 500 ppm
I20260809 05:35:14.438701 4876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:14.441897 4876 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:35:14.443410 4876 webserver.cc:460] Webserver started at http://127.30.137.195:37273/ using document root <none> and password file <none>
I20260809 05:35:14.444093 4876 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:14.444243 4876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:14.444566 4876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:14.447613 4876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "c37ef17e559e421fb7dfae102c93a964"
format_stamp: "Formatted at 2026-08-09 05:35:14 on dist-test-slave-3kk6"
server_key: "49bf27c193065f3acacc0f11662a4af0"
server_key_iv: "3bf889e67fb61da8aabd4be0c62c2809"
server_key_version: "encryptionkey@0"
I20260809 05:35:14.448436 4876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "c37ef17e559e421fb7dfae102c93a964"
format_stamp: "Formatted at 2026-08-09 05:35:14 on dist-test-slave-3kk6"
server_key: "49bf27c193065f3acacc0f11662a4af0"
server_key_iv: "3bf889e67fb61da8aabd4be0c62c2809"
server_key_version: "encryptionkey@0"
I20260809 05:35:14.453469 4876 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:35:14.456863 4656 heartbeater.cc:503] Master 127.30.137.254:33373 was elected leader, sending a full tablet report...
I20260809 05:35:14.456918 4906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:14.458470 4876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:35:14.458722 4876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "c37ef17e559e421fb7dfae102c93a964"
format_stamp: "Formatted at 2026-08-09 05:35:14 on dist-test-slave-3kk6"
server_key: "49bf27c193065f3acacc0f11662a4af0"
server_key_iv: "3bf889e67fb61da8aabd4be0c62c2809"
server_key_version: "encryptionkey@0"
I20260809 05:35:14.459018 4876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:14.470211 4876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:14.474208 4876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:14.474514 4876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:14.475471 4876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:14.476783 4876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:14.476859 4876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:14.477036 4876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:14.477090 4876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:14.511615 4876 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:32873
I20260809 05:35:14.511670 5072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:32873 every 8 connection(s)
I20260809 05:35:14.513041 4876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:14.518803 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 4876
I20260809 05:35:14.519018 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:14.519587 31271 external_mini_cluster.cc:1428] Setting key 63950debb92c7510e0e6253b4c0060da
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:14.534765 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.515781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:47665 (local address 127.30.137.254:33373)
0809 05:35:14.516293 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:14.516299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:14.517286 (+ 987us) server_negotiation.cc:407] Connection header received
0809 05:35:14.519573 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:14.519578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:14.519662 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:14.519807 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:14.521541 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.522814 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.524677 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.525065 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.525165 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.528904 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:14.528909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:14.528912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:14.528945 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:14.531720 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.532379 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.532383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.532385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.532512 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.532885 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.532887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.532889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.533143 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:14.533310 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:14.534070 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 05:35:14.534317 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":227,"threads_started":1}
I20260809 05:35:14.535805 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.516397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33373 (local address 127.30.137.195:47665)
0809 05:35:14.517096 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:14.517140 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:14.519261 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:14.520103 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:14.520115 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:14.520698 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:14.521346 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:14.521376 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.522966 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.522971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:14.524522 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:14.524540 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.525264 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.525270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:14.525469 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.526197 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:14.526226 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:14.528703 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:14.531840 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:14.531846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:14.531857 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:14.532272 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:14.532606 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:14.532610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:14.532612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:14.532780 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:14.533269 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:14.533287 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:14.533643 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 05:35:14.533937 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 05:35:14.534478 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":284,"threads_started":1}
I20260809 05:35:14.537122 5074 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33373
I20260809 05:35:14.537467 5074 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:14.538168 5074 heartbeater.cc:511] Master 127.30.137.254:33373 requested a full tablet report, sending...
I20260809 05:35:14.539601 4353 ts_manager.cc:194] Registered new tserver with Master: c37ef17e559e421fb7dfae102c93a964 (127.30.137.195:32873)
I20260809 05:35:14.540403 4353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:47665
I20260809 05:35:14.547585 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:14.605221 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.589782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.30.137.193:45993)
0809 05:35:14.590508 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:14.590516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:14.590543 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:35:14.590608 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:14.590614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:14.590902 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:14.591055 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:14.592550 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.593647 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.594939 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.595508 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.595608 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.599739 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:14.599752 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:14.599768 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:14.599804 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:14.602589 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.603307 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.603314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.603316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.603443 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.603768 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.603771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.603773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.604112 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:14.604355 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:14.604592 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 05:35:14.604791 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":359,"threads_started":1}
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:14.624462 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.609778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.30.137.254:33373)
0809 05:35:14.610056 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:14.610065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:14.610373 (+ 308us) server_negotiation.cc:407] Connection header received
0809 05:35:14.610583 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:14.610587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:14.610731 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:14.610839 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:14.612171 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.612927 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.614167 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.614581 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.614771 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.618773 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:14.618779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:14.618782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:14.618819 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:14.621530 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.622236 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.622244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.622252 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.622388 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.622861 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.622866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.622872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.623223 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:14.623485 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:14.623882 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 05:35:14.624127 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 05:35:14.625058 4353 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60044
I20260809 05:35:14.626081 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 4446
I20260809 05:35:14.638733 5087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.627029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.30.137.193:45993)
0809 05:35:14.627260 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:14.627268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:14.627526 (+ 258us) server_negotiation.cc:407] Connection header received
0809 05:35:14.627637 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:14.627641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:14.627779 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:14.627902 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:14.629087 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.629797 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.631058 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.631469 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.631569 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.633798 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:14.633802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:14.633805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:14.633831 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:14.636368 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.637033 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.637037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.637039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.637157 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.637479 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.637482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.637484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.637727 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:14.637907 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:14.638144 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 05:35:14.638319 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:35:14.639389 4645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:14.724884 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 4446
I20260809 05:35:14.744448 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 4665
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:35:14.760222 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.745428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42938 (local address 127.30.137.194:41125)
0809 05:35:14.746033 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:14.746041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:14.746067 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:35:14.746131 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:14.746137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:14.746367 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:14.746500 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:14.747766 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.748791 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.749961 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.750493 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.750589 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.754879 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:14.754890 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:14.754903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:14.754941 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:14.757670 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.758291 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.758298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.758300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.758424 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.758913 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.758916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.758918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.759255 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:14.759515 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:14.759732 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 05:35:14.759934 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":306,"threads_started":1}
I20260809 05:35:14.761541 4860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:14.846099 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 4665
I20260809 05:35:14.869006 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 4876
Aug 09 05:35:14 dist-test-slave-3kk6 krb5kdc[4283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:35:14.886935 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:14.870209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.30.137.195:32873)
0809 05:35:14.870956 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:14.870964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:14.870996 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:35:14.871074 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:14.871081 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:14.871290 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:14.871415 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:14.872696 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.873778 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.875171 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:14.875778 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:14.875898 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:14.880436 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:14.880449 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:14.880464 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:14.880503 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:14.883724 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.884530 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.884539 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.884541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.884675 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:14.885151 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:14.885157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:14.885161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:14.885535 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:14.885890 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:14.886248 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 05:35:14.886532 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":370,"threads_started":1}
I20260809 05:35:14.888324 5068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:14.973052 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 4876
I20260809 05:35:14.999588 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 4303
I20260809 05:35:15.000937 4401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:15.108270 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 4303
2026-08-09T05:35:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4387 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:15 dist-test-slave-3kk6 krb5kdc[5104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:15 dist-test-slave-3kk6 krb5kdc[5104](info): set up 2 sockets
Aug 09 05:35:15 dist-test-slave-3kk6 krb5kdc[5104](info): commencing operation
krb5kdc: starting...
W20260809 05:35:16.422405 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:16 dist-test-slave-3kk6 krb5kdc[5104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:16.664598 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:33255
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:33255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:17.001477 5122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:17.001798 5122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:17.001902 5122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:17.009205 5122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:17.009346 5122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:17.009395 5122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:17.009456 5122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:17.009506 5122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:17.019505 5122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:33255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:33255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:17.020809 5122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:17.022121 5122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:17.034188 5129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:17.034250 5135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:17.034240 5132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:17.035068 5122 server_base.cc:1034] running on GCE node
I20260809 05:35:17.035884 5122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:17.037315 5122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:17.038568 5122 hybrid_clock.cc:648] HybridClock initialized: now 1786253717038533 us; error 47 us; skew 500 ppm
I20260809 05:35:17.038955 5122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:17 dist-test-slave-3kk6 krb5kdc[5104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:17.042290 5122 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:17.043551 5122 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:17.043697 5122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:17.044059 5122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:17.047271 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "cbabac570d0d48188c6ea256940a991e"
format_stamp: "Formatted at 2026-08-09 05:35:17 on dist-test-slave-3kk6"
server_key: "0ff15057846cf28ca774c5a83cdf42b7"
server_key_iv: "bf14f8aff456b8a93ebee967af8472dc"
server_key_version: "encryptionkey@0"
I20260809 05:35:17.048153 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "cbabac570d0d48188c6ea256940a991e"
format_stamp: "Formatted at 2026-08-09 05:35:17 on dist-test-slave-3kk6"
server_key: "0ff15057846cf28ca774c5a83cdf42b7"
server_key_iv: "bf14f8aff456b8a93ebee967af8472dc"
server_key_version: "encryptionkey@0"
I20260809 05:35:17.053515 5122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:35:17.058033 5143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:17.059535 5122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:35:17.059685 5122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "cbabac570d0d48188c6ea256940a991e"
format_stamp: "Formatted at 2026-08-09 05:35:17 on dist-test-slave-3kk6"
server_key: "0ff15057846cf28ca774c5a83cdf42b7"
server_key_iv: "bf14f8aff456b8a93ebee967af8472dc"
server_key_version: "encryptionkey@0"
I20260809 05:35:17.059886 5122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:17.078594 5122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:17.083405 5122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:17.083648 5122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:17.102977 5122 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:33255
I20260809 05:35:17.102924 5216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:33255 every 8 connection(s)
I20260809 05:35:17.103955 5122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:17.107792 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:17.108140 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 5122
I20260809 05:35:17.108407 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:17.109268 31271 external_mini_cluster.cc:1428] Setting key 25db7a7dae46d8a68d5eef8216f5689d
I20260809 05:35:17.122505 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e: Bootstrap starting.
Aug 09 05:35:17 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:17.126492 5217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:17.127928 5217 log.cc:824] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:17.131376 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e: No bootstrap required, opened a new log
I20260809 05:35:17.134477 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:17.111542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.30.137.254:33255)
0809 05:35:17.112437 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:17.112448 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:17.112492 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:35:17.114381 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:17.114404 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:17.114880 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:17.115217 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:17.117074 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:17.118398 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:17.120300 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:17.120972 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:17.121102 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:17.126122 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:17.126135 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:17.126153 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:17.126210 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:17.130158 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:17.130952 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:17.130960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:17.130968 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:17.131188 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:17.131615 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:17.131619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:17.131621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:17.132221 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:17.132463 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:17.132956 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 05:35:17.133386 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":418,"threads_started":1}
I20260809 05:35:17.139621 5217 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabac570d0d48188c6ea256940a991e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33255 } }
I20260809 05:35:17.140024 5217 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:17.140137 5217 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbabac570d0d48188c6ea256940a991e, State: Initialized, Role: FOLLOWER
I20260809 05:35:17.140756 5217 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabac570d0d48188c6ea256940a991e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33255 } }
I20260809 05:35:17.141023 5217 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:17.141165 5217 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:17.141348 5217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:17.143748 5217 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabac570d0d48188c6ea256940a991e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33255 } }
I20260809 05:35:17.144261 5217 leader_election.cc:302] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbabac570d0d48188c6ea256940a991e; no voters:
I20260809 05:35:17.144853 5217 leader_election.cc:288] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:17.144979 5228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:17.146260 5228 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [term 1 LEADER]: Becoming Leader. State: Replica: cbabac570d0d48188c6ea256940a991e, State: Running, Role: LEADER
I20260809 05:35:17.146864 5228 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabac570d0d48188c6ea256940a991e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33255 } }
I20260809 05:35:17.147185 5217 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:17.149837 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbabac570d0d48188c6ea256940a991e. Latest consensus state: current_term: 1 leader_uuid: "cbabac570d0d48188c6ea256940a991e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabac570d0d48188c6ea256940a991e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33255 } } }
I20260809 05:35:17.150264 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:17.150523 5231 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbabac570d0d48188c6ea256940a991e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbabac570d0d48188c6ea256940a991e" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33255 } } }
I20260809 05:35:17.150904 5231 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:17.152413 5240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:17.157724 5240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:17.167233 5240 catalog_manager.cc:1347] Generated new cluster ID: 48d38e10699a49628b8fd0e70d0c225a
I20260809 05:35:17.167359 5240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:17.201833 5240 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:17.203327 5240 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:17.218194 5240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e: Generated new TSK 0
I20260809 05:35:17.219152 5240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:35:17.294215 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33255
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:17.641360 5261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:17.641772 5261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:17.641953 5261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:17.649772 5261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:17.649968 5261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:17.650151 5261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:17.661087 5261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:17.662556 5261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:17.664140 5261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:17.677719 5268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:17.677712 5267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:17.677953 5273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:17.678849 5261 server_base.cc:1034] running on GCE node
I20260809 05:35:17.679497 5261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:17.680536 5261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:17.681813 5261 hybrid_clock.cc:648] HybridClock initialized: now 1786253717681741 us; error 74 us; skew 500 ppm
I20260809 05:35:17.682183 5261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:17 dist-test-slave-3kk6 krb5kdc[5104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:17.689930 5261 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:35:17.691375 5261 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:17.691526 5261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:17.691885 5261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:17.695150 5261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "9628440cd8124a9b861d44635b229a35"
format_stamp: "Formatted at 2026-08-09 05:35:17 on dist-test-slave-3kk6"
server_key: "26e99ae6300bcaaa333f9e0d16ade36a"
server_key_iv: "cc6a08308b997a274c77da3308a58a35"
server_key_version: "encryptionkey@0"
I20260809 05:35:17.696061 5261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "9628440cd8124a9b861d44635b229a35"
format_stamp: "Formatted at 2026-08-09 05:35:17 on dist-test-slave-3kk6"
server_key: "26e99ae6300bcaaa333f9e0d16ade36a"
server_key_iv: "cc6a08308b997a274c77da3308a58a35"
server_key_version: "encryptionkey@0"
I20260809 05:35:17.701649 5261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:35:17.705550 5284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:17.707063 5261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:17.707260 5261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "9628440cd8124a9b861d44635b229a35"
format_stamp: "Formatted at 2026-08-09 05:35:17 on dist-test-slave-3kk6"
server_key: "26e99ae6300bcaaa333f9e0d16ade36a"
server_key_iv: "cc6a08308b997a274c77da3308a58a35"
server_key_version: "encryptionkey@0"
I20260809 05:35:17.707528 5261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:17.722581 5261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:17.726613 5261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:17.726969 5261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:17.727972 5261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:17.729420 5261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:17.729539 5261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:17.729674 5261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:17.729764 5261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:17.764400 5261 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:35561
I20260809 05:35:17.764477 5460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:35561 every 8 connection(s)
I20260809 05:35:17.765358 5261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:17.770380 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 5261
I20260809 05:35:17.770619 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:17.771275 31271 external_mini_cluster.cc:1428] Setting key 0cc3b0cc1a21e0801915b4273c87c940
Aug 09 05:35:17 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:17.787426 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:17.767824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:43995 (local address 127.30.137.254:33255)
0809 05:35:17.768288 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:17.768292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:17.769167 (+ 875us) server_negotiation.cc:407] Connection header received
0809 05:35:17.771169 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:17.771174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:17.771288 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:17.771413 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:17.773435 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:17.774547 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:17.776239 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:17.776949 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:17.777094 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:17.780857 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:17.780866 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:17.780874 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:17.780921 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:17.783927 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:17.784926 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:17.784930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:17.784932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:17.785057 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:17.785562 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:17.785566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:17.785568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:17.785847 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:17.785959 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:17.786974 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0809 05:35:17.787145 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":253,"threads_started":1}
I20260809 05:35:17.789069 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:17.768267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33255 (local address 127.30.137.193:43995)
0809 05:35:17.768992 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:17.769031 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:17.770896 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:17.771675 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:17.771683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:17.772395 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:17.773225 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:17.773255 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:17.774777 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:17.774781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:17.776073 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:17.776086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:17.777171 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:17.777175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:17.777363 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:17.778082 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:17.778110 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:17.780617 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:17.784083 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:17.784095 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:17.784115 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:17.784777 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:17.785170 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:17.785177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:17.785183 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:17.785439 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:17.785986 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:17.785998 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:17.786525 (+ 527us) client_negotiation.cc:769] Sending connection context
0809 05:35:17.786873 (+ 348us) client_negotiation.cc:240] Negotiation successful
0809 05:35:17.787522 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":292,"threads_started":1}
I20260809 05:35:17.790856 5463 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33255
I20260809 05:35:17.791173 5463 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:17.791875 5463 heartbeater.cc:511] Master 127.30.137.254:33255 requested a full tablet report, sending...
I20260809 05:35:17.794365 5172 ts_manager.cc:194] Registered new tserver with Master: 9628440cd8124a9b861d44635b229a35 (127.30.137.193:35561)
I20260809 05:35:17.796581 5172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:43995
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:35:17.830806 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33255
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:18.178228 5478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:18.178558 5478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:18.178774 5478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:18.186021 5478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:18.186189 5478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:18.186389 5478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:18.196583 5478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:18.198000 5478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:18.199440 5478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:18.212893 5486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:18.212831 5487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:18.212913 5489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:18.213861 5478 server_base.cc:1034] running on GCE node
I20260809 05:35:18.214517 5478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:18.215513 5478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:18.216717 5478 hybrid_clock.cc:648] HybridClock initialized: now 1786253718216650 us; error 62 us; skew 500 ppm
I20260809 05:35:18.217038 5478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:18 dist-test-slave-3kk6 krb5kdc[5104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:18.220331 5478 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:35:18.221406 5478 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:18.221501 5478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:18.221771 5478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:18.224752 5478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "d7bcc359cab24f668e3de209a0747201"
format_stamp: "Formatted at 2026-08-09 05:35:18 on dist-test-slave-3kk6"
server_key: "7ac3c572aa057c66d6f24dff8052a7ab"
server_key_iv: "e7fae19088f95c11ba5287ad91260afa"
server_key_version: "encryptionkey@0"
I20260809 05:35:18.225540 5478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "d7bcc359cab24f668e3de209a0747201"
format_stamp: "Formatted at 2026-08-09 05:35:18 on dist-test-slave-3kk6"
server_key: "7ac3c572aa057c66d6f24dff8052a7ab"
server_key_iv: "e7fae19088f95c11ba5287ad91260afa"
server_key_version: "encryptionkey@0"
I20260809 05:35:18.230713 5478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 05:35:18.234197 5495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:18.235529 5478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 05:35:18.235666 5478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "d7bcc359cab24f668e3de209a0747201"
format_stamp: "Formatted at 2026-08-09 05:35:18 on dist-test-slave-3kk6"
server_key: "7ac3c572aa057c66d6f24dff8052a7ab"
server_key_iv: "e7fae19088f95c11ba5287ad91260afa"
server_key_version: "encryptionkey@0"
I20260809 05:35:18.235857 5478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:18.257674 5478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:18.261857 5478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:18.262102 5478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:18.263047 5478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:18.264437 5478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:18.264531 5478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:18.264612 5478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:18.264729 5478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:18.300264 5478 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:44853
I20260809 05:35:18.300364 5668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:44853 every 8 connection(s)
I20260809 05:35:18.301216 5478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:18.310096 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 5478
I20260809 05:35:18.310328 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:18.310892 31271 external_mini_cluster.cc:1428] Setting key 50e9ef58802f564cfcd867d5aa788d81
Aug 09 05:35:18 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:18.322834 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.303977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:56769 (local address 127.30.137.254:33255)
0809 05:35:18.304472 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:18.304476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:18.305303 (+ 827us) server_negotiation.cc:407] Connection header received
0809 05:35:18.307447 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:18.307452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:18.307545 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:18.307671 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:18.309265 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.310003 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.311916 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.312444 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.312626 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.316660 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:18.316665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:18.316669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:18.316702 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:18.319558 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:18.320369 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:18.320374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:18.320376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:18.320516 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:18.320986 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:18.320990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:18.320992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:18.321278 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:18.321440 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:18.322164 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 05:35:18.322408 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":274,"threads_started":1}
I20260809 05:35:18.323851 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.304396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33255 (local address 127.30.137.194:56769)
0809 05:35:18.305131 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:18.305173 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:18.307160 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:18.307922 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:18.307931 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:18.308501 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:18.309078 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.309096 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.310163 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.310168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.311760 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.311774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.312731 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.312740 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.313064 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.313816 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:18.313839 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:18.316382 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:18.319734 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:18.319743 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:18.319761 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:18.320221 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:18.320636 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:18.320641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:18.320645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:18.320845 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:18.321421 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:18.321429 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:18.321786 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 05:35:18.322065 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 05:35:18.322670 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":308,"threads_started":1}
I20260809 05:35:18.325448 5669 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33255
I20260809 05:35:18.325811 5669 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:18.326624 5669 heartbeater.cc:511] Master 127.30.137.254:33255 requested a full tablet report, sending...
I20260809 05:35:18.328536 5172 ts_manager.cc:194] Registered new tserver with Master: d7bcc359cab24f668e3de209a0747201 (127.30.137.194:44853)
I20260809 05:35:18.329418 5172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:56769
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:35:18.367415 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33255
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:18.712952 5677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:18.713246 5677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:18.713349 5677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:18.720824 5677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:18.720937 5677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:18.721057 5677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:18.730943 5677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:18.732216 5677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:18.733649 5677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:18.746917 5690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:18.746956 5692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:18.747023 5688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:18.751387 5677 server_base.cc:1034] running on GCE node
I20260809 05:35:18.752028 5677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:18.753011 5677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:18.754215 5677 hybrid_clock.cc:648] HybridClock initialized: now 1786253718754159 us; error 50 us; skew 500 ppm
I20260809 05:35:18.754585 5677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:18 dist-test-slave-3kk6 krb5kdc[5104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:18.757848 5677 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:35:18.759225 5677 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:18.759366 5677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:18.759721 5677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:18.762964 5677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "72ffb03f37484a118643877ed4d34565"
format_stamp: "Formatted at 2026-08-09 05:35:18 on dist-test-slave-3kk6"
server_key: "4fadafd39a74b43fee522062466ab45f"
server_key_iv: "0dc84e610a73cead4238ed3141485b88"
server_key_version: "encryptionkey@0"
I20260809 05:35:18.763919 5677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "72ffb03f37484a118643877ed4d34565"
format_stamp: "Formatted at 2026-08-09 05:35:18 on dist-test-slave-3kk6"
server_key: "4fadafd39a74b43fee522062466ab45f"
server_key_iv: "0dc84e610a73cead4238ed3141485b88"
server_key_version: "encryptionkey@0"
I20260809 05:35:18.769353 5677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:18.773063 5699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:18.774364 5677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 05:35:18.774549 5677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "72ffb03f37484a118643877ed4d34565"
format_stamp: "Formatted at 2026-08-09 05:35:18 on dist-test-slave-3kk6"
server_key: "4fadafd39a74b43fee522062466ab45f"
server_key_iv: "0dc84e610a73cead4238ed3141485b88"
server_key_version: "encryptionkey@0"
I20260809 05:35:18.774847 5677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:18.787040 5677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:18.791483 5677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:18.791831 5677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:18.792837 5677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:18.794284 5677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:18.794416 5677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:18.794543 5677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:18.794633 5677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:18.799925 5463 heartbeater.cc:503] Master 127.30.137.254:33255 was elected leader, sending a full tablet report...
I20260809 05:35:18.828532 5677 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:33761
I20260809 05:35:18.828613 5867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:33761 every 8 connection(s)
I20260809 05:35:18.829385 5677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:18.831784 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 5677
I20260809 05:35:18.832016 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:18.832592 31271 external_mini_cluster.cc:1428] Setting key 658785f9b05e9e15c4780a486c409e75
Aug 09 05:35:18 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:18.851127 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.832131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:46459 (local address 127.30.137.254:33255)
0809 05:35:18.832701 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:18.832709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:18.833884 (+ 1175us) server_negotiation.cc:407] Connection header received
0809 05:35:18.835931 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:18.835938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:18.836027 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:18.836119 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:18.837714 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.838532 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.840529 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.840956 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.841067 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.844720 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:18.844725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:18.844728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:18.844762 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:18.848011 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:18.848971 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:18.848975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:18.848977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:18.849099 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:18.849503 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:18.849506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:18.849508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:18.849789 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:18.849907 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:18.850602 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 05:35:18.850858 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":295,"threads_started":1}
I20260809 05:35:18.852188 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.832774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33255 (local address 127.30.137.195:46459)
0809 05:35:18.833701 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:18.833770 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:18.835672 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:18.836331 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:18.836338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:18.836912 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:18.837506 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.837525 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.838747 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.838752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.840366 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.840380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.841159 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.841164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.841361 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.842113 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:18.842137 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:18.844494 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:18.848160 (+ 3666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:18.848169 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:18.848187 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:18.848848 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:18.849198 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:18.849202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:18.849204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:18.849403 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:18.849937 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:18.849943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:18.850270 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 05:35:18.850507 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 05:35:18.851019 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":419,"threads_started":1}
I20260809 05:35:18.853637 5868 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33255
I20260809 05:35:18.853953 5868 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:18.854592 5868 heartbeater.cc:511] Master 127.30.137.254:33255 requested a full tablet report, sending...
I20260809 05:35:18.856037 5172 ts_manager.cc:194] Registered new tserver with Master: 72ffb03f37484a118643877ed4d34565 (127.30.137.195:33761)
I20260809 05:35:18.856848 5172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:46459
I20260809 05:35:18.860776 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:35:18.878312 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.865407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.30.137.254:33255)
0809 05:35:18.865626 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:18.865635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:18.866115 (+ 480us) server_negotiation.cc:407] Connection header received
0809 05:35:18.866255 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:18.866259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:18.866321 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:18.866415 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:18.867928 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.868631 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.869900 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.870330 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.870425 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.872754 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:18.872759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:18.872762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:18.872799 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:18.875752 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:18.876338 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:18.876342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:18.876344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:18.876461 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:18.876834 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:18.876838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:18.876840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:18.877185 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:18.877386 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:18.877701 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 05:35:18.877900 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:35:18.882472 5172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:18.889494 5172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:18.929911 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.902774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:35561 (local address 127.0.0.1:54934)
0809 05:35:18.916282 (+ 13508us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:18.916301 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:18.917968 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:18.921440 (+ 3472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:18.921450 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:18.921482 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:18.921966 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.921982 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.924105 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.924111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.926903 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.926919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.928824 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.928832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.929046 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.929113 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 05:35:18.929235 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 05:35:18.929431 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13385,"mutex_wait_us":1,"thread_start_us":13968,"threads_started":1}
I20260809 05:35:18.930328 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.903160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54934 (local address 127.30.137.193:35561)
0809 05:35:18.920547 (+ 17387us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:18.920558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:18.920588 (+ 30us) server_negotiation.cc:407] Connection header received
0809 05:35:18.920691 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:18.920699 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:18.920954 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:18.922221 (+ 1267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:18.922468 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.923950 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.927078 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.928586 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.929599 (+ 1013us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.929772 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:18.929855 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 05:35:18.930033 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17192,"thread_start_us":18342,"threads_started":1}
I20260809 05:35:18.930512 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.902818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.30.137.194:44853)
0809 05:35:18.918876 (+ 16058us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:18.918890 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:18.918932 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:35:18.919066 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:18.919079 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:18.919439 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:18.919702 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:18.923228 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.924943 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.927455 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.928515 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.928680 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.928882 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:18.929975 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0809 05:35:18.930145 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15782,"thread_start_us":18265,"threads_started":1}
I20260809 05:35:18.930801 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.902500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:44853 (local address 127.0.0.1:33482)
0809 05:35:18.916145 (+ 13645us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:18.916238 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:18.916514 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:18.922217 (+ 5703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:18.922225 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:18.922265 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:18.922976 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.922989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.925127 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.925132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.927297 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.927307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.929047 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.929055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.929268 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.929307 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 05:35:18.929393 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 05:35:18.929520 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13220,"thread_start_us":13876,"threads_started":1}
I20260809 05:35:18.931380 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.917771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.30.137.195:33761)
0809 05:35:18.922237 (+ 4466us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:18.922245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:18.923101 (+ 856us) server_negotiation.cc:407] Connection header received
0809 05:35:18.923264 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:18.923271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:18.923506 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:18.923714 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:18.924786 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.925993 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.928386 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.929846 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.930043 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.930195 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:18.930824 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 05:35:18.930999 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4280,"thread_start_us":4082,"threads_started":1}
I20260809 05:35:18.931759 5896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:18.918240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:33761 (local address 127.0.0.1:36576)
0809 05:35:18.922951 (+ 4711us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:18.922974 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:18.923109 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:18.923762 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:18.923767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:18.923792 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:18.924513 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.924525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.926165 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.926169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.928192 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:18.928208 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:18.930338 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:18.930345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:18.930523 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:18.930565 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 05:35:18.931290 (+ 725us) client_negotiation.cc:240] Negotiation successful
0809 05:35:18.931425 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4549,"thread_start_us":4267,"threads_started":1}
I20260809 05:35:18.934763 5567 tablet_service.cc:1467] Processing CreateTablet for tablet df68a0171ad5494e946e72ea207e50a2 (DEFAULT_TABLE table=test-table [id=7a441f26e45f44848305f0163303e4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:18.934602 5362 tablet_service.cc:1467] Processing CreateTablet for tablet df68a0171ad5494e946e72ea207e50a2 (DEFAULT_TABLE table=test-table [id=7a441f26e45f44848305f0163303e4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:18.935382 5770 tablet_service.cc:1467] Processing CreateTablet for tablet df68a0171ad5494e946e72ea207e50a2 (DEFAULT_TABLE table=test-table [id=7a441f26e45f44848305f0163303e4ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:18.936035 5567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df68a0171ad5494e946e72ea207e50a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:18.936141 5362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df68a0171ad5494e946e72ea207e50a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:18.936733 5770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df68a0171ad5494e946e72ea207e50a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:18.950382 5897 tablet_bootstrap.cc:492] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Bootstrap starting.
I20260809 05:35:18.952890 5900 tablet_bootstrap.cc:492] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Bootstrap starting.
I20260809 05:35:18.953485 5899 tablet_bootstrap.cc:492] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Bootstrap starting.
I20260809 05:35:18.955099 5897 tablet_bootstrap.cc:654] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:18.956259 5899 tablet_bootstrap.cc:654] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:18.956503 5897 log.cc:824] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:18.956673 5900 tablet_bootstrap.cc:654] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:18.957446 5899 log.cc:824] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:18.958026 5900 log.cc:824] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:18.960911 5897 tablet_bootstrap.cc:492] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: No bootstrap required, opened a new log
I20260809 05:35:18.961053 5900 tablet_bootstrap.cc:492] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: No bootstrap required, opened a new log
I20260809 05:35:18.961229 5897 ts_tablet_manager.cc:1397] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 05:35:18.961360 5899 tablet_bootstrap.cc:492] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: No bootstrap required, opened a new log
I20260809 05:35:18.961356 5900 ts_tablet_manager.cc:1397] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 05:35:18.961632 5899 ts_tablet_manager.cc:1397] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 05:35:18.969710 5900 raft_consensus.cc:348] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:18.970180 5900 raft_consensus.cc:374] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:18.970325 5900 raft_consensus.cc:729] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ffb03f37484a118643877ed4d34565, State: Initialized, Role: FOLLOWER
I20260809 05:35:18.971024 5900 consensus_queue.cc:260] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:18.972255 5868 heartbeater.cc:503] Master 127.30.137.254:33255 was elected leader, sending a full tablet report...
I20260809 05:35:18.972806 5900 ts_tablet_manager.cc:1428] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260809 05:35:18.972765 5899 raft_consensus.cc:348] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:18.973218 5899 raft_consensus.cc:374] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:18.973333 5899 raft_consensus.cc:729] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9628440cd8124a9b861d44635b229a35, State: Initialized, Role: FOLLOWER
I20260809 05:35:18.973961 5899 consensus_queue.cc:260] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:18.974272 5897 raft_consensus.cc:348] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:18.974865 5897 raft_consensus.cc:374] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:18.975016 5897 raft_consensus.cc:729] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7bcc359cab24f668e3de209a0747201, State: Initialized, Role: FOLLOWER
I20260809 05:35:18.975644 5897 consensus_queue.cc:260] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:18.976089 5899 ts_tablet_manager.cc:1428] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260809 05:35:18.983001 5669 heartbeater.cc:503] Master 127.30.137.254:33255 was elected leader, sending a full tablet report...
I20260809 05:35:18.983850 5897 ts_tablet_manager.cc:1428] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Time spent starting tablet: real 0.022s user 0.017s sys 0.004s
I20260809 05:35:19.002568 5904 raft_consensus.cc:482] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:19.002946 5904 raft_consensus.cc:504] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:19.004626 5904 leader_election.cc:288] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7bcc359cab24f668e3de209a0747201 (127.30.137.194:44853), 9628440cd8124a9b861d44635b229a35 (127.30.137.193:35561)
I20260809 05:35:19.011746 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.005319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:35561 (local address 127.30.137.195:36625)
0809 05:35:19.005556 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:19.005584 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:19.005705 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:19.006167 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:19.006173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:19.006232 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:19.006462 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:19.006476 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.007809 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.007813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:19.009745 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:19.009761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.011008 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.011012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:19.011261 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.011286 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 05:35:19.011364 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 05:35:19.011519 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":27}
I20260809 05:35:19.011848 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.005317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:36625 (local address 127.30.137.193:35561)
0809 05:35:19.005527 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:19.005533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:19.005675 (+ 142us) server_negotiation.cc:407] Connection header received
0809 05:35:19.005867 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:19.005876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:19.005999 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:19.006118 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:19.006639 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.007675 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.009909 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.010870 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.011034 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.011107 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:19.011422 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 05:35:19.011563 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:35:19.012866 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.005232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:55539 (local address 127.30.137.194:44853)
0809 05:35:19.005467 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:19.005476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:19.006495 (+ 1019us) server_negotiation.cc:407] Connection header received
0809 05:35:19.006678 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:19.006683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:19.006766 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:19.006869 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:19.007361 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.008354 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.010528 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.011736 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.011905 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.011969 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:19.012517 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 05:35:19.012661 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 05:35:19.012967 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.005332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:44853 (local address 127.30.137.195:55539)
0809 05:35:19.006361 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:19.006391 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:19.006515 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:19.006893 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:19.006897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:19.006918 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:19.007205 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:19.007213 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.008520 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.008526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:19.010372 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:19.010384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.012067 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.012074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:19.012366 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.012390 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:35:19.012465 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 05:35:19.012548 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":198,"thread_start_us":457,"threads_started":1}
I20260809 05:35:19.013266 5395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df68a0171ad5494e946e72ea207e50a2" candidate_uuid: "72ffb03f37484a118643877ed4d34565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9628440cd8124a9b861d44635b229a35" is_pre_election: true
I20260809 05:35:19.013885 5395 raft_consensus.cc:2454] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ffb03f37484a118643877ed4d34565 in term 0.
I20260809 05:35:19.014042 5595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df68a0171ad5494e946e72ea207e50a2" candidate_uuid: "72ffb03f37484a118643877ed4d34565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7bcc359cab24f668e3de209a0747201" is_pre_election: true
I20260809 05:35:19.014568 5595 raft_consensus.cc:2454] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ffb03f37484a118643877ed4d34565 in term 0.
I20260809 05:35:19.014627 5704 leader_election.cc:302] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ffb03f37484a118643877ed4d34565, 9628440cd8124a9b861d44635b229a35; no voters:
I20260809 05:35:19.015156 5904 raft_consensus.cc:2781] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:19.015329 5904 raft_consensus.cc:482] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:19.015447 5904 raft_consensus.cc:3037] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 0 FOLLOWER]: Advancing to term 1
W20260809 05:35:19.017652 5464 tablet.cc:2369] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:19.017716 5904 raft_consensus.cc:504] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:19.018496 5904 leader_election.cc:288] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [CANDIDATE]: Term 1 election: Requested vote from peers d7bcc359cab24f668e3de209a0747201 (127.30.137.194:44853), 9628440cd8124a9b861d44635b229a35 (127.30.137.193:35561)
I20260809 05:35:19.018965 5595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df68a0171ad5494e946e72ea207e50a2" candidate_uuid: "72ffb03f37484a118643877ed4d34565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7bcc359cab24f668e3de209a0747201"
I20260809 05:35:19.019167 5595 raft_consensus.cc:3037] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:19.019109 5395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df68a0171ad5494e946e72ea207e50a2" candidate_uuid: "72ffb03f37484a118643877ed4d34565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9628440cd8124a9b861d44635b229a35"
I20260809 05:35:19.019366 5395 raft_consensus.cc:3037] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:19.021468 5595 raft_consensus.cc:2454] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ffb03f37484a118643877ed4d34565 in term 1.
I20260809 05:35:19.021562 5395 raft_consensus.cc:2454] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ffb03f37484a118643877ed4d34565 in term 1.
I20260809 05:35:19.021885 5702 leader_election.cc:302] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ffb03f37484a118643877ed4d34565, d7bcc359cab24f668e3de209a0747201; no voters:
I20260809 05:35:19.022197 5904 raft_consensus.cc:2781] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:19.022727 5904 raft_consensus.cc:686] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 1 LEADER]: Becoming Leader. State: Replica: 72ffb03f37484a118643877ed4d34565, State: Running, Role: LEADER
I20260809 05:35:19.023203 5904 consensus_queue.cc:237] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } }
I20260809 05:35:19.027590 5172 catalog_manager.cc:5495] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72ffb03f37484a118643877ed4d34565 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "72ffb03f37484a118643877ed4d34565" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ffb03f37484a118643877ed4d34565" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33761 } health_report { overall_health: HEALTHY } } }
W20260809 05:35:19.053330 5670 tablet.cc:2369] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:19.055780 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.049564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.30.137.195:33761)
0809 05:35:19.049842 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:19.049850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:19.050002 (+ 152us) server_negotiation.cc:407] Connection header received
0809 05:35:19.050308 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:19.050313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:19.050397 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:19.050560 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:19.050996 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.051790 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.053358 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.053903 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.054006 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.054595 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:19.055161 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:19.055282 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:19.055421 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 05:35:19.055492 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 05:35:19.064965 5395 raft_consensus.cc:1261] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 1 FOLLOWER]: Refusing update from remote peer 72ffb03f37484a118643877ed4d34565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:19.064968 5595 raft_consensus.cc:1261] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 1 FOLLOWER]: Refusing update from remote peer 72ffb03f37484a118643877ed4d34565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:19.070930 5904 consensus_queue.cc:1035] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9628440cd8124a9b861d44635b229a35" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:19.071456 5914 consensus_queue.cc:1035] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7bcc359cab24f668e3de209a0747201" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:35:19.080984 5869 tablet.cc:2369] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:19.100127 5172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:19.100399 5172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:19.104343 5172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbabac570d0d48188c6ea256940a991e: Sending DeleteTablet for 3 replicas of tablet df68a0171ad5494e946e72ea207e50a2
I20260809 05:35:19.105469 5362 tablet_service.cc:1514] Processing DeleteTablet for tablet df68a0171ad5494e946e72ea207e50a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:19 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54934
I20260809 05:35:19.105566 5567 tablet_service.cc:1514] Processing DeleteTablet for tablet df68a0171ad5494e946e72ea207e50a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:19 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:33482
I20260809 05:35:19.112604 5770 tablet_service.cc:1514] Processing DeleteTablet for tablet df68a0171ad5494e946e72ea207e50a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:19 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36576
I20260809 05:35:19.116611 5936 tablet_replica.cc:330] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: stopping tablet replica
I20260809 05:35:19.117260 5936 raft_consensus.cc:2229] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:19.117883 5936 raft_consensus.cc:2258] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:19.118503 5934 tablet_replica.cc:330] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: stopping tablet replica
I20260809 05:35:19.119313 5934 raft_consensus.cc:2229] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:19.119961 5934 raft_consensus.cc:2258] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:19.122414 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 5261
I20260809 05:35:19.123618 5938 tablet_replica.cc:330] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: stopping tablet replica
I20260809 05:35:19.123996 5938 raft_consensus.cc:2229] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:19.124658 5938 raft_consensus.cc:2258] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:19.127228 5938 ts_tablet_manager.cc:1905] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:19.132468 5936 ts_tablet_manager.cc:1905] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:19.134862 5938 ts_tablet_manager.cc:1918] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:19.135110 5938 log.cc:1196] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/df68a0171ad5494e946e72ea207e50a2
I20260809 05:35:19.135792 5938 ts_tablet_manager.cc:1939] T df68a0171ad5494e946e72ea207e50a2 P 72ffb03f37484a118643877ed4d34565: Deleting consensus metadata
I20260809 05:35:19.136992 5934 ts_tablet_manager.cc:1905] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:19.137809 5150 catalog_manager.cc:4841] TS 72ffb03f37484a118643877ed4d34565 (127.30.137.195:33761): tablet df68a0171ad5494e946e72ea207e50a2 (table test-table [id=7a441f26e45f44848305f0163303e4ea]) successfully deleted
I20260809 05:35:19.143949 5936 ts_tablet_manager.cc:1918] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:19.144199 5936 log.cc:1196] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/df68a0171ad5494e946e72ea207e50a2
I20260809 05:35:19.144843 5936 ts_tablet_manager.cc:1939] T df68a0171ad5494e946e72ea207e50a2 P d7bcc359cab24f668e3de209a0747201: Deleting consensus metadata
I20260809 05:35:19.145736 5934 ts_tablet_manager.cc:1918] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:19.146003 5934 log.cc:1196] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/df68a0171ad5494e946e72ea207e50a2
I20260809 05:35:19.146944 5148 catalog_manager.cc:4841] TS d7bcc359cab24f668e3de209a0747201 (127.30.137.194:44853): tablet df68a0171ad5494e946e72ea207e50a2 (table test-table [id=7a441f26e45f44848305f0163303e4ea]) successfully deleted
I20260809 05:35:19.147594 5934 ts_tablet_manager.cc:1939] T df68a0171ad5494e946e72ea207e50a2 P 9628440cd8124a9b861d44635b229a35: Deleting consensus metadata
I20260809 05:35:19.149462 5150 catalog_manager.cc:4841] TS 9628440cd8124a9b861d44635b229a35 (127.30.137.193:35561): tablet df68a0171ad5494e946e72ea207e50a2 (table test-table [id=7a441f26e45f44848305f0163303e4ea]) successfully deleted
Aug 09 05:35:19 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:19.157887 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.124279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.30.137.193:35561)
0809 05:35:19.124448 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:19.124457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:19.135928 (+ 11471us) server_negotiation.cc:407] Connection header received
0809 05:35:19.136264 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:19.136271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:19.136389 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:19.136469 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:19.138310 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.139488 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.144627 (+ 5139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.145339 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.145457 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.150731 (+ 5274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:19.150747 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:19.150767 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:19.150820 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:19.154880 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:19.155573 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:19.155583 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:19.155587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:19.155802 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:19.156177 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:19.156183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:19.156186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:19.156678 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:19.157003 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:19.157252 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 05:35:19.157498 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 05:35:19.158601 5452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:19.250547 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 5261
I20260809 05:35:19.285740 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 5478
Aug 09 05:35:19 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:35:19.304200 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.286690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.30.137.194:44853)
0809 05:35:19.286870 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:19.286875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:19.286894 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:35:19.287086 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:19.287091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:19.287200 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:19.287295 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:19.288980 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.289731 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.291624 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.292032 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.292131 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.297644 (+ 5513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:19.297662 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:19.297678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:19.297727 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:19.300986 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:19.301742 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:19.301750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:19.301752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:19.301905 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:19.302326 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:19.302330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:19.302332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:19.302827 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:19.303144 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:19.303461 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 05:35:19.303718 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:35:19.304939 5662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:19.397382 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 5478
I20260809 05:35:19.419845 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 5677
Aug 09 05:35:19 dist-test-slave-3kk6 krb5kdc[5104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:35:19.438252 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:19.420800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36606 (local address 127.30.137.195:33761)
0809 05:35:19.421037 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:19.421043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:19.421099 (+ 56us) server_negotiation.cc:407] Connection header received
0809 05:35:19.421178 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:19.421186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:19.421309 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:19.421427 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:19.422698 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.423968 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.425224 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:19.425973 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:19.426133 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:19.430694 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:19.430713 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:19.430733 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:19.430795 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:19.435107 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:19.435900 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:19.435916 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:19.435920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:19.436131 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:19.436507 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:19.436513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:19.436517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:19.437048 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:19.437381 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:19.437557 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 05:35:19.437811 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 05:35:19.439178 5860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:19.565636 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 5677
I20260809 05:35:19.595830 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 5122
I20260809 05:35:19.597119 5211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:19.707957 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 5122
2026-08-09T05:35:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4599 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T05:35:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:19Z Disabled control of system clock
I20260809 05:35:19.792224 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43437
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 05:35:20.149725 5957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:20.150053 5957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:20.150141 5957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:20.157585 5957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:20.157707 5957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:20.157747 5957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:20.157805 5957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:20.157850 5957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:20.167542 5957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:20.168860 5957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:20.171139 5957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:20.183352 5971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:20.183418 5969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:20.183414 5966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:20.184566 5957 server_base.cc:1034] running on GCE node
I20260809 05:35:20.185436 5957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:20.187019 5957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:20.188283 5957 hybrid_clock.cc:648] HybridClock initialized: now 1786253720188243 us; error 53 us; skew 500 ppm
I20260809 05:35:20.188690 5957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:20.191303 5957 webserver.cc:460] Webserver started at http://127.30.137.254:38481/ using document root <none> and password file <none>
I20260809 05:35:20.192086 5957 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:20.192236 5957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:20.192595 5957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:20.195811 5957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "9972992ac6bf4d9f8c6ea2866422db97"
format_stamp: "Formatted at 2026-08-09 05:35:20 on dist-test-slave-3kk6"
server_key: "108e340e45eb44782197f8c8c1c17845"
server_key_iv: "45b0043c0f483a4cda8d554347d5ff82"
server_key_version: "encryptionkey@0"
I20260809 05:35:20.196738 5957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "9972992ac6bf4d9f8c6ea2866422db97"
format_stamp: "Formatted at 2026-08-09 05:35:20 on dist-test-slave-3kk6"
server_key: "108e340e45eb44782197f8c8c1c17845"
server_key_iv: "45b0043c0f483a4cda8d554347d5ff82"
server_key_version: "encryptionkey@0"
I20260809 05:35:20.202368 5957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:20.206104 5981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:20.207724 5957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:20.207942 5957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "9972992ac6bf4d9f8c6ea2866422db97"
format_stamp: "Formatted at 2026-08-09 05:35:20 on dist-test-slave-3kk6"
server_key: "108e340e45eb44782197f8c8c1c17845"
server_key_iv: "45b0043c0f483a4cda8d554347d5ff82"
server_key_version: "encryptionkey@0"
I20260809 05:35:20.208202 5957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:20.214506 5957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:20.215389 5957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:20.215744 5957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:20.235093 5957 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:43437
I20260809 05:35:20.235128 6066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:43437 every 8 connection(s)
I20260809 05:35:20.236677 5957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:20.238044 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 5957
I20260809 05:35:20.238271 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:20.239048 31271 external_mini_cluster.cc:1428] Setting key 3aa41e246fc16e520bbdd2e2ebeb526f
I20260809 05:35:20.241319 6069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:20.248847 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:20.241271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.30.137.254:43437)
0809 05:35:20.242433 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:20.242448 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:20.242501 (+ 53us) server_negotiation.cc:407] Connection header received
0809 05:35:20.244620 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:20.244648 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:20.245207 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:20.245674 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:20.246779 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:20.246788 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:20.246805 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:20.246823 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:20.246920 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:20.246943 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:20.247177 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:20.247336 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 05:35:20.247602 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":398,"threads_started":1}
I20260809 05:35:20.254915 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97: Bootstrap starting.
I20260809 05:35:20.257870 6069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:20.259009 6069 log.cc:824] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:20.261771 6069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97: No bootstrap required, opened a new log
I20260809 05:35:20.269127 6069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9972992ac6bf4d9f8c6ea2866422db97" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43437 } }
I20260809 05:35:20.269555 6069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:20.269672 6069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972992ac6bf4d9f8c6ea2866422db97, State: Initialized, Role: FOLLOWER
I20260809 05:35:20.270258 6069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9972992ac6bf4d9f8c6ea2866422db97" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43437 } }
I20260809 05:35:20.270526 6069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:20.270726 6069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:20.270955 6069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:20.273456 6069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9972992ac6bf4d9f8c6ea2866422db97" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43437 } }
I20260809 05:35:20.274004 6069 leader_election.cc:302] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9972992ac6bf4d9f8c6ea2866422db97; no voters:
I20260809 05:35:20.274638 6069 leader_election.cc:288] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:20.274888 6078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:20.275718 6078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [term 1 LEADER]: Becoming Leader. State: Replica: 9972992ac6bf4d9f8c6ea2866422db97, State: Running, Role: LEADER
I20260809 05:35:20.276430 6078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9972992ac6bf4d9f8c6ea2866422db97" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43437 } }
I20260809 05:35:20.276959 6069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:20.279385 6079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9972992ac6bf4d9f8c6ea2866422db97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9972992ac6bf4d9f8c6ea2866422db97" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43437 } } }
I20260809 05:35:20.279853 6079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:20.279803 6081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9972992ac6bf4d9f8c6ea2866422db97. Latest consensus state: current_term: 1 leader_uuid: "9972992ac6bf4d9f8c6ea2866422db97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9972992ac6bf4d9f8c6ea2866422db97" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43437 } } }
I20260809 05:35:20.280238 6081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:20.282469 6090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:20.289197 6090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:20.296100 6100 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 05:35:20.306937 6090 catalog_manager.cc:1347] Generated new cluster ID: 1af631f703c44c5eb1e35ab13d89bbde
I20260809 05:35:20.307088 6090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:20.325151 6090 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:20.325671 6090 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:20.340276 6090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97: Generated new TSK 0
I20260809 05:35:20.341444 6090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:35:20.353506 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43437
--builtin_ntp_servers=127.30.137.212:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 05:35:20.710208 6103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:20.710515 6103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:20.710721 6103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:20.718575 6103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:20.718748 6103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:20.718940 6103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:20.728698 6103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:20.730022 6103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:20.731827 6103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:20.745723 6111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:20.745779 6115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:20.745658 6113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:20.746894 6103 server_base.cc:1034] running on GCE node
I20260809 05:35:20.747574 6103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:20.748637 6103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:20.749852 6103 hybrid_clock.cc:648] HybridClock initialized: now 1786253720749809 us; error 44 us; skew 500 ppm
I20260809 05:35:20.750202 6103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:20.752928 6103 webserver.cc:460] Webserver started at http://127.30.137.193:35573/ using document root <none> and password file <none>
I20260809 05:35:20.753700 6103 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:20.753827 6103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:20.754201 6103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:20.757359 6103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "ad2d3cea7e484b7f83c66078925596b1"
format_stamp: "Formatted at 2026-08-09 05:35:20 on dist-test-slave-3kk6"
server_key: "3c2dc1d6f8d8814d7c30cb6a63c6f8aa"
server_key_iv: "dfaed6d7db2eda6c7bda8046af9bdb19"
server_key_version: "encryptionkey@0"
I20260809 05:35:20.758231 6103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "ad2d3cea7e484b7f83c66078925596b1"
format_stamp: "Formatted at 2026-08-09 05:35:20 on dist-test-slave-3kk6"
server_key: "3c2dc1d6f8d8814d7c30cb6a63c6f8aa"
server_key_iv: "dfaed6d7db2eda6c7bda8046af9bdb19"
server_key_version: "encryptionkey@0"
I20260809 05:35:20.763934 6103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 05:35:20.767753 6124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:20.769222 6103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:35:20.769423 6103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "ad2d3cea7e484b7f83c66078925596b1"
format_stamp: "Formatted at 2026-08-09 05:35:20 on dist-test-slave-3kk6"
server_key: "3c2dc1d6f8d8814d7c30cb6a63c6f8aa"
server_key_iv: "dfaed6d7db2eda6c7bda8046af9bdb19"
server_key_version: "encryptionkey@0"
I20260809 05:35:20.769678 6103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:20.775930 6103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:20.776916 6103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:20.777230 6103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:20.778395 6103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:20.780160 6103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:20.780305 6103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:20.780444 6103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:20.780536 6103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:20.816624 6103 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:39761
I20260809 05:35:20.816690 6297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:39761 every 8 connection(s)
I20260809 05:35:20.818049 6103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:20.820288 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 6103
I20260809 05:35:20.820505 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:20.821406 31271 external_mini_cluster.cc:1428] Setting key 1607ebfcd2f2ab67561ae14049ecd280
I20260809 05:35:20.826177 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43437
--builtin_ntp_servers=127.30.137.212:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 05:35:20.828392 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:20.820985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:39131 (local address 127.30.137.254:43437)
0809 05:35:20.821506 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:20.821514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:20.823326 (+ 1812us) server_negotiation.cc:407] Connection header received
0809 05:35:20.825702 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:20.825708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:20.825780 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:20.825907 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:20.827411 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:20.827419 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:20.827427 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:20.827452 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:20.827512 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:20.827531 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:20.827676 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:20.827911 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 05:35:20.827982 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":272,"threads_started":1}
I20260809 05:35:20.829272 6301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:20.821765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43437 (local address 127.30.137.193:39131)
0809 05:35:20.823106 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:20.823161 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:20.825403 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:20.826119 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:20.826128 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:20.826241 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:20.827081 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:20.827084 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:20.827124 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:20.827220 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:20.827676 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:20.827687 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:20.827695 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 05:35:20.827946 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 05:35:20.828165 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":392,"threads_started":1}
I20260809 05:35:20.831004 6298 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43437
I20260809 05:35:20.831336 6298 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:20.831993 6298 heartbeater.cc:511] Master 127.30.137.254:43437 requested a full tablet report, sending...
I20260809 05:35:20.834273 6009 ts_manager.cc:194] Registered new tserver with Master: ad2d3cea7e484b7f83c66078925596b1 (127.30.137.193:39761)
W20260809 05:35:21.186064 6302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:21.186401 6302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:21.186556 6302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:21.194350 6302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:21.194510 6302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:21.194716 6302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:21.204500 6302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:21.206017 6302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:21.208057 6302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:21.222970 6309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:21.222985 6314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:21.222887 6317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:21.224434 6302 server_base.cc:1034] running on GCE node
I20260809 05:35:21.225201 6302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:21.226224 6302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:21.227480 6302 hybrid_clock.cc:648] HybridClock initialized: now 1786253721227434 us; error 44 us; skew 500 ppm
I20260809 05:35:21.227875 6302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:21.230507 6302 webserver.cc:460] Webserver started at http://127.30.137.194:34297/ using document root <none> and password file <none>
I20260809 05:35:21.231334 6302 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:21.231488 6302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:21.231880 6302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:21.235110 6302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "3e7a63dad6fd497bb628a59fce6abe52"
format_stamp: "Formatted at 2026-08-09 05:35:21 on dist-test-slave-3kk6"
server_key: "25874a66340b00f78bdd2ac1e1eee757"
server_key_iv: "b741ad378d536adaab2e5fdcf1d41d73"
server_key_version: "encryptionkey@0"
I20260809 05:35:21.236004 6302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "3e7a63dad6fd497bb628a59fce6abe52"
format_stamp: "Formatted at 2026-08-09 05:35:21 on dist-test-slave-3kk6"
server_key: "25874a66340b00f78bdd2ac1e1eee757"
server_key_iv: "b741ad378d536adaab2e5fdcf1d41d73"
server_key_version: "encryptionkey@0"
I20260809 05:35:21.242050 6302 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 05:35:21.246157 6327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:21.247781 6302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:21.247987 6302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "3e7a63dad6fd497bb628a59fce6abe52"
format_stamp: "Formatted at 2026-08-09 05:35:21 on dist-test-slave-3kk6"
server_key: "25874a66340b00f78bdd2ac1e1eee757"
server_key_iv: "b741ad378d536adaab2e5fdcf1d41d73"
server_key_version: "encryptionkey@0"
I20260809 05:35:21.248239 6302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:21.254359 6302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:21.255297 6302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:21.255601 6302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:21.256579 6302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:21.258129 6302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:21.258260 6302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:21.258411 6302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:21.258505 6302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:21.294443 6302 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:44295
I20260809 05:35:21.294503 6496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:44295 every 8 connection(s)
I20260809 05:35:21.296166 6302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:21.296717 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 6302
I20260809 05:35:21.296940 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:21.297623 31271 external_mini_cluster.cc:1428] Setting key 0fad604c1e212adda1f700ebcbc4cd7d
I20260809 05:35:21.300877 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43437
--builtin_ntp_servers=127.30.137.212:36647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 05:35:21.306070 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.299520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:33381 (local address 127.30.137.254:43437)
0809 05:35:21.299713 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.299719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.301612 (+ 1893us) server_negotiation.cc:407] Connection header received
0809 05:35:21.303498 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.303506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.303606 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.303731 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.305182 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.305191 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.305198 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.305229 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.305283 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.305302 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.305437 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.305627 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.305702 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:35:21.307078 6500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.300287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43437 (local address 127.30.137.194:33381)
0809 05:35:21.301415 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.301477 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.303199 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.303956 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.303963 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.304058 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.304826 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.304829 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.304869 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.304961 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.305414 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.305424 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.305429 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.305662 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.305885 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":365,"threads_started":1}
I20260809 05:35:21.308583 6498 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43437
I20260809 05:35:21.308938 6498 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:21.309715 6498 heartbeater.cc:511] Master 127.30.137.254:43437 requested a full tablet report, sending...
I20260809 05:35:21.311494 6009 ts_manager.cc:194] Registered new tserver with Master: 3e7a63dad6fd497bb628a59fce6abe52 (127.30.137.194:44295)
W20260809 05:35:21.662432 6501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:21.662868 6501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:21.662973 6501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:21.670636 6501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:21.670853 6501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:21.670975 6501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:21.680787 6501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:21.682364 6501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:21.684345 6501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:21.698501 6507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:21.698938 6510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:21.698982 6506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:21.699565 6501 server_base.cc:1034] running on GCE node
I20260809 05:35:21.700273 6501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:21.701339 6501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:21.702553 6501 hybrid_clock.cc:648] HybridClock initialized: now 1786253721702508 us; error 37 us; skew 500 ppm
I20260809 05:35:21.702927 6501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:21.705384 6501 webserver.cc:460] Webserver started at http://127.30.137.195:38633/ using document root <none> and password file <none>
I20260809 05:35:21.706144 6501 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:21.706271 6501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:21.706701 6501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:21.709934 6501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "bba34445dfa54ecebbfe3b47a3a96eda"
format_stamp: "Formatted at 2026-08-09 05:35:21 on dist-test-slave-3kk6"
server_key: "748dc3363de347ac8c354f5b53e20509"
server_key_iv: "53c5376a1e32db9771d30e39f20a581d"
server_key_version: "encryptionkey@0"
I20260809 05:35:21.710881 6501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "bba34445dfa54ecebbfe3b47a3a96eda"
format_stamp: "Formatted at 2026-08-09 05:35:21 on dist-test-slave-3kk6"
server_key: "748dc3363de347ac8c354f5b53e20509"
server_key_iv: "53c5376a1e32db9771d30e39f20a581d"
server_key_version: "encryptionkey@0"
I20260809 05:35:21.716876 6501 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 05:35:21.721105 6521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:21.722865 6501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:35:21.723079 6501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "bba34445dfa54ecebbfe3b47a3a96eda"
format_stamp: "Formatted at 2026-08-09 05:35:21 on dist-test-slave-3kk6"
server_key: "748dc3363de347ac8c354f5b53e20509"
server_key_iv: "53c5376a1e32db9771d30e39f20a581d"
server_key_version: "encryptionkey@0"
I20260809 05:35:21.723366 6501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:21.729766 6501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:21.730793 6501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:21.731107 6501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:21.732164 6501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:21.733777 6501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:21.733892 6501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:21.734040 6501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:21.734133 6501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:21.771885 6501 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:39175
I20260809 05:35:21.771993 6684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:39175 every 8 connection(s)
I20260809 05:35:21.773514 6501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:21.779951 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 6501
I20260809 05:35:21.780107 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:21.780591 31271 external_mini_cluster.cc:1428] Setting key 5ea7e91c17c96d86a61f657179c82f23
I20260809 05:35:21.783962 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.776609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:33237 (local address 127.30.137.254:43437)
0809 05:35:21.776824 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.776829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.778601 (+ 1772us) server_negotiation.cc:407] Connection header received
0809 05:35:21.780853 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.780860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.780972 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.781103 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.783035 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.783041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.783047 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.783074 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.783111 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.783125 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.783264 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.783520 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.783608 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:35:21.785065 6687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.777291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43437 (local address 127.30.137.195:33237)
0809 05:35:21.778358 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.778443 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.780469 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.781437 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.781489 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.781625 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.782584 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.782594 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.782688 (+ 94us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.782834 (+ 146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.783237 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.783250 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.783260 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.783619 (+ 359us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.783933 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":312,"threads_started":1}
I20260809 05:35:21.786494 6685 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43437
I20260809 05:35:21.786942 6685 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:21.787679 6685 heartbeater.cc:511] Master 127.30.137.254:43437 requested a full tablet report, sending...
I20260809 05:35:21.789284 6009 ts_manager.cc:194] Registered new tserver with Master: bba34445dfa54ecebbfe3b47a3a96eda (127.30.137.195:39175)
I20260809 05:35:21.796944 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:35:21.804946 6300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.802082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.30.137.254:43437)
0809 05:35:21.802294 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.802300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.802947 (+ 647us) server_negotiation.cc:407] Connection header received
0809 05:35:21.803108 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.803112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.803175 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.803275 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.804280 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.804287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.804290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.804308 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.804343 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.804357 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.804476 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.804641 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.804695 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 05:35:21.809623 6009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:21.817101 6009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:21.837646 6298 heartbeater.cc:503] Master 127.30.137.254:43437 was elected leader, sending a full tablet report...
I20260809 05:35:21.855372 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.832300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49490 (local address 127.30.137.193:39761)
0809 05:35:21.849651 (+ 17351us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.849665 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.849726 (+ 61us) server_negotiation.cc:407] Connection header received
0809 05:35:21.849832 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.849846 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.850204 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.850437 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.854105 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.854117 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.854132 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.854157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.854237 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.854265 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.854471 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.854794 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.854916 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17140,"thread_start_us":16985,"threads_started":1}
I20260809 05:35:21.855398 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.831600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49490)
0809 05:35:21.848578 (+ 16978us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.848610 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.848783 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.852582 (+ 3799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.852591 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.852698 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.853758 (+ 1060us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.853763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.853798 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.853893 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.854532 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.854545 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.854553 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.854795 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.854902 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16752,"thread_start_us":20587,"threads_started":1}
I20260809 05:35:21.856649 6704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.832825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:44295 (local address 127.0.0.1:48932)
0809 05:35:21.852505 (+ 19680us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.852528 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.852628 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.853855 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.853862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.853886 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.854584 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.854586 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.854599 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.854637 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.855971 (+ 1334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.855977 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.855983 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.856108 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.856167 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19529,"thread_start_us":16069,"threads_started":1}
I20260809 05:35:21.857158 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.833398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.30.137.194:44295)
0809 05:35:21.853196 (+ 19798us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.853207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.853242 (+ 35us) server_negotiation.cc:407] Connection header received
0809 05:35:21.853311 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.853317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.853593 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.854326 (+ 733us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.855723 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.855732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.855750 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.855773 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.855819 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.855835 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.856635 (+ 800us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.856724 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.856831 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19593,"thread_start_us":19456,"threads_started":1}
I20260809 05:35:21.858076 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.830923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:39175 (local address 127.0.0.1:41132)
0809 05:35:21.832470 (+ 1547us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.832536 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.832718 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.854443 (+ 21725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.854449 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.854474 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.856322 (+ 1848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.856326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.856343 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.856392 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.857424 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.857429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.857434 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.857580 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.857634 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"thread_start_us":966,"threads_started":1}
I20260809 05:35:21.858084 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.831125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.30.137.195:39175)
0809 05:35:21.853776 (+ 22651us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.853784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.853825 (+ 41us) server_negotiation.cc:407] Connection header received
0809 05:35:21.853909 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.853913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.854111 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.856029 (+ 1918us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.857187 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.857196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.857211 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.857233 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.857296 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.857321 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.857510 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.857582 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.857680 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22455,"spinlock_wait_cycles":23424,"thread_start_us":22434,"threads_started":1}
I20260809 05:35:21.860822 6399 tablet_service.cc:1467] Processing CreateTablet for tablet cf35002f48fd44bb8ec7ee9576ee53a8 (DEFAULT_TABLE table=test-table [id=7b05b21b46654661aab76def520b4218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:21.860960 6204 tablet_service.cc:1467] Processing CreateTablet for tablet cf35002f48fd44bb8ec7ee9576ee53a8 (DEFAULT_TABLE table=test-table [id=7b05b21b46654661aab76def520b4218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:21.862432 6204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf35002f48fd44bb8ec7ee9576ee53a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:21.862581 6399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf35002f48fd44bb8ec7ee9576ee53a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:21.862164 6588 tablet_service.cc:1467] Processing CreateTablet for tablet cf35002f48fd44bb8ec7ee9576ee53a8 (DEFAULT_TABLE table=test-table [id=7b05b21b46654661aab76def520b4218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:21.863679 6588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf35002f48fd44bb8ec7ee9576ee53a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:21.875728 6710 tablet_bootstrap.cc:492] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Bootstrap starting.
I20260809 05:35:21.879343 6711 tablet_bootstrap.cc:492] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Bootstrap starting.
I20260809 05:35:21.879766 6715 tablet_bootstrap.cc:492] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: Bootstrap starting.
I20260809 05:35:21.880275 6710 tablet_bootstrap.cc:654] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:21.881814 6710 log.cc:824] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:21.883553 6711 tablet_bootstrap.cc:654] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:21.884061 6715 tablet_bootstrap.cc:654] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:21.885030 6711 log.cc:824] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:21.885252 6715 log.cc:824] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:21.887007 6710 tablet_bootstrap.cc:492] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: No bootstrap required, opened a new log
I20260809 05:35:21.887328 6710 ts_tablet_manager.cc:1397] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260809 05:35:21.888296 6715 tablet_bootstrap.cc:492] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: No bootstrap required, opened a new log
I20260809 05:35:21.888588 6715 ts_tablet_manager.cc:1397] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260809 05:35:21.888602 6711 tablet_bootstrap.cc:492] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: No bootstrap required, opened a new log
I20260809 05:35:21.888896 6711 ts_tablet_manager.cc:1397] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 05:35:21.896643 6715 raft_consensus.cc:348] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.897415 6715 raft_consensus.cc:374] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:21.897557 6715 raft_consensus.cc:729] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2d3cea7e484b7f83c66078925596b1, State: Initialized, Role: FOLLOWER
I20260809 05:35:21.898351 6715 consensus_queue.cc:260] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.898819 6710 raft_consensus.cc:348] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.899390 6710 raft_consensus.cc:374] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:21.899540 6710 raft_consensus.cc:729] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e7a63dad6fd497bb628a59fce6abe52, State: Initialized, Role: FOLLOWER
I20260809 05:35:21.900146 6711 raft_consensus.cc:348] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.900555 6715 ts_tablet_manager.cc:1428] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260809 05:35:21.900384 6710 consensus_queue.cc:260] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.900700 6711 raft_consensus.cc:374] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:21.900876 6711 raft_consensus.cc:729] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bba34445dfa54ecebbfe3b47a3a96eda, State: Initialized, Role: FOLLOWER
I20260809 05:35:21.901734 6711 consensus_queue.cc:260] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.903266 6685 heartbeater.cc:503] Master 127.30.137.254:43437 was elected leader, sending a full tablet report...
I20260809 05:35:21.905905 6711 ts_tablet_manager.cc:1428] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
I20260809 05:35:21.908354 6498 heartbeater.cc:503] Master 127.30.137.254:43437 was elected leader, sending a full tablet report...
I20260809 05:35:21.909497 6710 ts_tablet_manager.cc:1428] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Time spent starting tablet: real 0.022s user 0.009s sys 0.011s
I20260809 05:35:21.924665 6727 raft_consensus.cc:482] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:21.925073 6727 raft_consensus.cc:504] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.927160 6727 leader_election.cc:288] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bba34445dfa54ecebbfe3b47a3a96eda (127.30.137.195:39175), 3e7a63dad6fd497bb628a59fce6abe52 (127.30.137.194:44295)
I20260809 05:35:21.931475 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.927623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:34269 (local address 127.30.137.195:39175)
0809 05:35:21.927868 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.927872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.928385 (+ 513us) server_negotiation.cc:407] Connection header received
0809 05:35:21.929034 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.929039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.929112 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.929249 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.930580 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.930595 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.930600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.930627 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.930698 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.930734 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.930878 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.931037 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.931150 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:35:21.931684 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.928836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:45051 (local address 127.30.137.194:44295)
0809 05:35:21.929069 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.929074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.929350 (+ 276us) server_negotiation.cc:407] Connection header received
0809 05:35:21.929428 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.929433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.929501 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.929633 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.930881 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.930887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.930891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.930916 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.930960 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.930978 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.931119 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.931317 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.931388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 05:35:21.931684 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.927491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:39175 (local address 127.30.137.193:34269)
0809 05:35:21.928230 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.928253 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.928409 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.929378 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.929386 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.929414 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.930303 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.930308 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.930330 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.930387 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.930860 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.930865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.930870 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.931030 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.931098 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":389,"threads_started":1}
I20260809 05:35:21.931957 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.928490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:44295 (local address 127.30.137.193:45051)
0809 05:35:21.929173 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:21.929194 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:21.929322 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:21.929661 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:21.929664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:21.929682 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:21.930575 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:35:21.930579 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:21.930597 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:35:21.930644 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:21.931118 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:21.931123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:21.931129 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:35:21.931336 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 05:35:21.931392 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":304,"threads_started":1}
I20260809 05:35:21.932963 6615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35002f48fd44bb8ec7ee9576ee53a8" candidate_uuid: "ad2d3cea7e484b7f83c66078925596b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" is_pre_election: true
I20260809 05:35:21.932981 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35002f48fd44bb8ec7ee9576ee53a8" candidate_uuid: "ad2d3cea7e484b7f83c66078925596b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e7a63dad6fd497bb628a59fce6abe52" is_pre_election: true
I20260809 05:35:21.933629 6423 raft_consensus.cc:2454] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2d3cea7e484b7f83c66078925596b1 in term 0.
I20260809 05:35:21.933629 6615 raft_consensus.cc:2454] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2d3cea7e484b7f83c66078925596b1 in term 0.
I20260809 05:35:21.934412 6128 leader_election.cc:302] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2d3cea7e484b7f83c66078925596b1, bba34445dfa54ecebbfe3b47a3a96eda; no voters:
I20260809 05:35:21.934906 6727 raft_consensus.cc:2781] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:21.935088 6727 raft_consensus.cc:482] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:21.935217 6727 raft_consensus.cc:3037] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:21.937654 6727 raft_consensus.cc:504] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.938390 6727 leader_election.cc:288] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [CANDIDATE]: Term 1 election: Requested vote from peers bba34445dfa54ecebbfe3b47a3a96eda (127.30.137.195:39175), 3e7a63dad6fd497bb628a59fce6abe52 (127.30.137.194:44295)
I20260809 05:35:21.939101 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35002f48fd44bb8ec7ee9576ee53a8" candidate_uuid: "ad2d3cea7e484b7f83c66078925596b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e7a63dad6fd497bb628a59fce6abe52"
I20260809 05:35:21.939103 6615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf35002f48fd44bb8ec7ee9576ee53a8" candidate_uuid: "ad2d3cea7e484b7f83c66078925596b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bba34445dfa54ecebbfe3b47a3a96eda"
I20260809 05:35:21.939356 6615 raft_consensus.cc:3037] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:21.939355 6423 raft_consensus.cc:3037] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:21.942241 6423 raft_consensus.cc:2454] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2d3cea7e484b7f83c66078925596b1 in term 1.
I20260809 05:35:21.942265 6615 raft_consensus.cc:2454] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2d3cea7e484b7f83c66078925596b1 in term 1.
I20260809 05:35:21.942898 6128 leader_election.cc:302] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2d3cea7e484b7f83c66078925596b1, bba34445dfa54ecebbfe3b47a3a96eda; no voters:
I20260809 05:35:21.943243 6727 raft_consensus.cc:2781] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:21.943748 6727 raft_consensus.cc:686] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 1 LEADER]: Becoming Leader. State: Replica: ad2d3cea7e484b7f83c66078925596b1, State: Running, Role: LEADER
I20260809 05:35:21.944308 6727 consensus_queue.cc:237] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } }
I20260809 05:35:21.949332 6008 catalog_manager.cc:5495] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2d3cea7e484b7f83c66078925596b1 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "ad2d3cea7e484b7f83c66078925596b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad2d3cea7e484b7f83c66078925596b1" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39761 } health_report { overall_health: HEALTHY } } }
I20260809 05:35:21.967986 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:21.965374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49496 (local address 127.30.137.193:39761)
0809 05:35:21.965562 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:21.965568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:21.965639 (+ 71us) server_negotiation.cc:407] Connection header received
0809 05:35:21.965905 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:21.965909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:21.965994 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:21.966114 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:21.967334 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:21.967340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:21.967345 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:21.967366 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:21.967409 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:21.967426 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:21.967553 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:21.967618 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 05:35:21.967695 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:35:21.977053 6423 raft_consensus.cc:1261] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 1 FOLLOWER]: Refusing update from remote peer ad2d3cea7e484b7f83c66078925596b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:21.977090 6615 raft_consensus.cc:1261] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 1 FOLLOWER]: Refusing update from remote peer ad2d3cea7e484b7f83c66078925596b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:21.978235 6735 consensus_queue.cc:1035] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e7a63dad6fd497bb628a59fce6abe52" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:21.978693 6727 consensus_queue.cc:1035] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bba34445dfa54ecebbfe3b47a3a96eda" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:21.996861 6740 mvcc.cc:205] Tried to move back new op lower bound from 7316495245206425600 to 7316495245088141312. Current Snapshot: MvccSnapshot[applied={T|T < 7316495245206425600}]
I20260809 05:35:22.008766 6008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:22.009107 6008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:22.013114 6008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9972992ac6bf4d9f8c6ea2866422db97: Sending DeleteTablet for 3 replicas of tablet cf35002f48fd44bb8ec7ee9576ee53a8
I20260809 05:35:22.014679 6204 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35002f48fd44bb8ec7ee9576ee53a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:22 UTC) from {username='slave'} at 127.0.0.1:49490
I20260809 05:35:22.015071 6588 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35002f48fd44bb8ec7ee9576ee53a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:22 UTC) from {username='slave'} at 127.0.0.1:41132
I20260809 05:35:22.015949 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 6103
I20260809 05:35:22.015908 6399 tablet_service.cc:1514] Processing DeleteTablet for tablet cf35002f48fd44bb8ec7ee9576ee53a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:22 UTC) from {username='slave'} at 127.0.0.1:48932
I20260809 05:35:22.019492 6754 tablet_replica.cc:330] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: stopping tablet replica
I20260809 05:35:22.019989 6754 raft_consensus.cc:2229] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:22.020545 6754 raft_consensus.cc:2258] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:22.023094 6754 ts_tablet_manager.cc:1905] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:22.023201 6751 tablet_replica.cc:330] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: stopping tablet replica
I20260809 05:35:22.023617 6751 raft_consensus.cc:2229] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:22.024326 6751 raft_consensus.cc:2258] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:22.025533 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.017594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49510 (local address 127.30.137.193:39761)
0809 05:35:22.017856 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:22.017865 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:22.018842 (+ 977us) server_negotiation.cc:407] Connection header received
0809 05:35:22.019110 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:22.019119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:22.019211 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:22.019338 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:22.024257 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:22.024266 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:22.024274 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:22.024323 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:22.024379 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:22.024406 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:22.024573 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:22.025151 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 05:35:22.025222 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:35:22.025146 6752 tablet_replica.cc:330] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: stopping tablet replica
I20260809 05:35:22.026572 6752 raft_consensus.cc:2229] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:22.026579 6751 ts_tablet_manager.cc:1905] T cf35002f48fd44bb8ec7ee9576ee53a8 P ad2d3cea7e484b7f83c66078925596b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:22.026712 6292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:22.027562 6752 raft_consensus.cc:2258] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:22.029786 6752 ts_tablet_manager.cc:1905] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:22.031103 6754 ts_tablet_manager.cc:1918] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:22.031292 6754 log.cc:1196] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/cf35002f48fd44bb8ec7ee9576ee53a8
I20260809 05:35:22.031951 6754 ts_tablet_manager.cc:1939] T cf35002f48fd44bb8ec7ee9576ee53a8 P 3e7a63dad6fd497bb628a59fce6abe52: Deleting consensus metadata
I20260809 05:35:22.034974 5985 catalog_manager.cc:4841] TS 3e7a63dad6fd497bb628a59fce6abe52 (127.30.137.194:44295): tablet cf35002f48fd44bb8ec7ee9576ee53a8 (table test-table [id=7b05b21b46654661aab76def520b4218]) successfully deleted
I20260809 05:35:22.037539 6752 ts_tablet_manager.cc:1918] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:22.037741 6752 log.cc:1196] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/cf35002f48fd44bb8ec7ee9576ee53a8
I20260809 05:35:22.038319 6752 ts_tablet_manager.cc:1939] T cf35002f48fd44bb8ec7ee9576ee53a8 P bba34445dfa54ecebbfe3b47a3a96eda: Deleting consensus metadata
I20260809 05:35:22.040372 5985 catalog_manager.cc:4841] TS bba34445dfa54ecebbfe3b47a3a96eda (127.30.137.195:39175): tablet cf35002f48fd44bb8ec7ee9576ee53a8 (table test-table [id=7b05b21b46654661aab76def520b4218]) successfully deleted
I20260809 05:35:22.133208 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 6103
W20260809 05:35:22.161284 5986 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.137.193:39761 (error 108)
I20260809 05:35:22.161473 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 6302
I20260809 05:35:22.163612 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.163020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49516)
0809 05:35:22.163244 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.163396 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 05:35:22.163872 5986 catalog_manager.cc:4584] TS ad2d3cea7e484b7f83c66078925596b1 (127.30.137.193:39761): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf35002f48fd44bb8ec7ee9576ee53a8: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
I20260809 05:35:22.165127 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.162368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.30.137.194:44295)
0809 05:35:22.162554 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:22.162558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:22.162732 (+ 174us) server_negotiation.cc:407] Connection header received
0809 05:35:22.162940 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:22.162946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:22.163063 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:22.163185 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:22.164501 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:22.164506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:22.164509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:22.164526 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:22.164557 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:22.164569 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:22.164698 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:22.164770 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 05:35:22.164816 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:35:22.165920 6489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:22.217761 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.217236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49530)
0809 05:35:22.217468 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.217580 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 05:35:22.218937 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.218522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49538)
0809 05:35:22.218731 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.218821 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 05:35:22.266484 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 6302
I20260809 05:35:22.279757 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.279213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49554)
0809 05:35:22.279434 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.279590 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 05:35:22.280903 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.280558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49566)
0809 05:35:22.280696 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.280770 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 05:35:22.293565 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 6501
I20260809 05:35:22.296802 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.294418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.30.137.195:39175)
0809 05:35:22.294609 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:22.294617 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:22.294706 (+ 89us) server_negotiation.cc:407] Connection header received
0809 05:35:22.294985 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:22.294989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:22.295081 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:22.295199 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:22.296105 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:22.296111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:22.296116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:22.296142 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:22.296175 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:22.296187 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:22.296331 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:22.296427 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 05:35:22.296494 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:35:22.297463 6679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:22.361418 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.360922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49568)
0809 05:35:22.361142 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.361259 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 05:35:22.362586 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:22.362154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39761 (local address 127.0.0.1:49576)
0809 05:35:22.362394 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:22.362469 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 05:35:22.389636 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 6501
I20260809 05:35:22.415866 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 5957
I20260809 05:35:22.417186 6060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:22.527164 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 5957
2026-08-09T05:35:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2821 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 05:35:22.581554 31271 mini_oidc.cc:152] Starting JWKS server
I20260809 05:35:22.584726 31271 webserver.cc:460] Webserver started at https://127.0.0.1:42365/ using document root <none> and password file <none>
I20260809 05:35:22.585143 31271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 05:35:22.585857 31271 webserver.cc:460] Webserver started at http://127.0.0.1:41425/ using document root <none> and password file <none>
2026-08-09T05:35:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:22Z Disabled control of system clock
I20260809 05:35:22.624634 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:41361
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:41361
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42365/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/testchainca.cert with env {}
W20260809 05:35:22.979079 6774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:22.979467 6774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:22.979643 6774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:22.987236 6774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:22.987411 6774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:22.987506 6774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:22.987622 6774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:22.987700 6774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 05:35:22.987794 6774 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 05:35:22.987874 6774 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42365/jwks.json
I20260809 05:35:22.998282 6774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:41361
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:41361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42365/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:22.999781 6774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:23.001232 6774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:23.014077 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:23.014108 6788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:23.014154 6791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:23.015206 6774 server_base.cc:1034] running on GCE node
I20260809 05:35:23.016069 6774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:23.017580 6774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:23.018805 6774 hybrid_clock.cc:648] HybridClock initialized: now 1786253723018761 us; error 41 us; skew 500 ppm
I20260809 05:35:23.019182 6774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:23.025698 6774 webserver.cc:460] Webserver started at http://127.30.137.254:41007/ using document root <none> and password file <none>
I20260809 05:35:23.026547 6774 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:23.026743 6774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:23.027129 6774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:23.030344 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "d5afd771d21b47d5b99c88f77c994566"
format_stamp: "Formatted at 2026-08-09 05:35:23 on dist-test-slave-3kk6"
server_key: "9284e1c6a0ede37872f589881369bf33"
server_key_iv: "afe4e92cf30391558422f802ceb9e17f"
server_key_version: "encryptionkey@0"
I20260809 05:35:23.031365 6774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "d5afd771d21b47d5b99c88f77c994566"
format_stamp: "Formatted at 2026-08-09 05:35:23 on dist-test-slave-3kk6"
server_key: "9284e1c6a0ede37872f589881369bf33"
server_key_iv: "afe4e92cf30391558422f802ceb9e17f"
server_key_version: "encryptionkey@0"
I20260809 05:35:23.037099 6774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:35:23.041133 6798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:23.042819 6774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:23.043089 6774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "d5afd771d21b47d5b99c88f77c994566"
format_stamp: "Formatted at 2026-08-09 05:35:23 on dist-test-slave-3kk6"
server_key: "9284e1c6a0ede37872f589881369bf33"
server_key_iv: "afe4e92cf30391558422f802ceb9e17f"
server_key_version: "encryptionkey@0"
I20260809 05:35:23.043414 6774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:23.069200 6774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:23.070266 6774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:23.070721 6774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:23.092669 6774 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:41361
I20260809 05:35:23.092700 6875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:41361 every 8 connection(s)
I20260809 05:35:23.094309 6774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:23.098811 6877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:23.101527 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 6774
I20260809 05:35:23.101794 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:23.102502 31271 external_mini_cluster.cc:1428] Setting key b8aecbec8ac7c95258dfa3a239439519
I20260809 05:35:23.111527 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566: Bootstrap starting.
I20260809 05:35:23.115142 6877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:23.116312 6877 log.cc:824] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:23.118227 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:23.104769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.30.137.254:41361)
0809 05:35:23.105908 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:23.105921 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:23.105973 (+ 52us) server_negotiation.cc:407] Connection header received
0809 05:35:23.107914 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:23.107947 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:23.108518 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:23.109009 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:23.109681 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.111091 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.112675 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.113428 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.113658 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:23.115081 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:23.115090 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:23.115105 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:23.115788 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:23.115889 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:23.115912 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:23.116059 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:23.116338 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 05:35:23.116730 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":433,"threads_started":1}
I20260809 05:35:23.120366 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566: No bootstrap required, opened a new log
I20260809 05:35:23.123016 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:35:23.128860 6877 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5afd771d21b47d5b99c88f77c994566" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41361 } }
I20260809 05:35:23.129284 6877 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:23.129396 6877 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5afd771d21b47d5b99c88f77c994566, State: Initialized, Role: FOLLOWER
I20260809 05:35:23.130031 6877 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5afd771d21b47d5b99c88f77c994566" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41361 } }
I20260809 05:35:23.130282 6877 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:23.130429 6877 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:23.130617 6877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:23.133294 6877 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5afd771d21b47d5b99c88f77c994566" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41361 } }
I20260809 05:35:23.133874 6877 leader_election.cc:302] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5afd771d21b47d5b99c88f77c994566; no voters:
I20260809 05:35:23.134501 6877 leader_election.cc:288] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:23.134747 6883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:23.135612 6883 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [term 1 LEADER]: Becoming Leader. State: Replica: d5afd771d21b47d5b99c88f77c994566, State: Running, Role: LEADER
I20260809 05:35:23.136216 6883 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5afd771d21b47d5b99c88f77c994566" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41361 } }
I20260809 05:35:23.136863 6877 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:23.139322 6884 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5afd771d21b47d5b99c88f77c994566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5afd771d21b47d5b99c88f77c994566" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41361 } } }
I20260809 05:35:23.139753 6884 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:23.139981 6885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5afd771d21b47d5b99c88f77c994566. Latest consensus state: current_term: 1 leader_uuid: "d5afd771d21b47d5b99c88f77c994566" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5afd771d21b47d5b99c88f77c994566" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41361 } } }
I20260809 05:35:23.140329 6885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:23.141394 6892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:23.149112 6892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:23.161056 6892 catalog_manager.cc:1347] Generated new cluster ID: 32d403b9c8594fd38397bceb523930b2
I20260809 05:35:23.161633 6892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:23.185456 6892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:23.187237 6892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:23.195387 6892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5afd771d21b47d5b99c88f77c994566: Generated new TSK 0
I20260809 05:35:23.196348 6892 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 05:35:23.246815 6920 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 05:35:23.247088 6920 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 05:35:23.249364 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:23.224733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.30.137.254:41361)
0809 05:35:23.224964 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:23.224971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:23.225676 (+ 705us) server_negotiation.cc:407] Connection header received
0809 05:35:23.226119 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:23.226127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:23.226225 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:23.226365 (+ 140us) server_negotiation.cc:226] Negotiated authn=JWT
0809 05:35:23.227115 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.227913 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.230128 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.230596 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.230871 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:23.231633 (+ 762us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 05:35:23.248595 (+ 16962us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 05:35:23.249113 (+ 518us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42834: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 05:35:23.249519 6881 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42834: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 05:35:23.251670 6918 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:23.224596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:41361 (local address 127.0.0.1:42834)
0809 05:35:23.225528 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:23.225557 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:23.225784 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:23.226438 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:23.226446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:23.226461 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0809 05:35:23.226933 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:23.226950 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.228091 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.228098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:23.229944 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:23.229962 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.230930 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.230937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:23.231146 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:23.231274 (+ 128us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 05:35:23.248984 (+ 17710us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 05:35:23.249447 (+ 463us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.137.254:41361: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":515,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.137.254:41361: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 05:35:23.253650 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 6774
I20260809 05:35:23.254933 6867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:23.353578 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 6774
2026-08-09T05:35:23Z chronyd exiting
I20260809 05:35:23.414929 31271 test_util.cc:175] -----------------------------------------------
I20260809 05:35:23.415083 31271 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786253675866932-31271-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (839 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 05:35:23.419018 31271 mini_oidc.cc:152] Starting JWKS server
I20260809 05:35:23.420922 31271 webserver.cc:460] Webserver started at https://127.0.0.1:42147/ using document root <none> and password file <none>
I20260809 05:35:23.421119 31271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 05:35:23.421766 31271 webserver.cc:460] Webserver started at http://127.0.0.1:45369/ using document root <none> and password file <none>
2026-08-09T05:35:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:23Z Disabled control of system clock
I20260809 05:35:23.459069 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39815
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:40125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:39815
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42147/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/testchainca.cert with env {}
W20260809 05:35:23.810096 6932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:23.810418 6932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:23.810585 6932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:23.818209 6932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:23.818332 6932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:23.818430 6932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:23.818540 6932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:23.818630 6932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 05:35:23.818768 6932 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 05:35:23.818820 6932 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42147/jwks.json
I20260809 05:35:23.829169 6932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:39815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42147/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:23.830549 6932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:23.831980 6932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:23.844317 6941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:23.844364 6942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:23.844419 6944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:23.845449 6932 server_base.cc:1034] running on GCE node
I20260809 05:35:23.846249 6932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:23.847779 6932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:23.849007 6932 hybrid_clock.cc:648] HybridClock initialized: now 1786253723848948 us; error 41 us; skew 500 ppm
I20260809 05:35:23.849351 6932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:23.851709 6932 webserver.cc:460] Webserver started at http://127.30.137.254:39841/ using document root <none> and password file <none>
I20260809 05:35:23.852387 6932 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:23.852481 6932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:23.852793 6932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:23.855909 6932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "8769e6a577044422ba2bfbbba5a9b4f5"
format_stamp: "Formatted at 2026-08-09 05:35:23 on dist-test-slave-3kk6"
server_key: "4b9451d6c81ecdae289d3539e7ca3daf"
server_key_iv: "8cc8c4f4af30f08055e86b17e92aefeb"
server_key_version: "encryptionkey@0"
I20260809 05:35:23.856832 6932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "8769e6a577044422ba2bfbbba5a9b4f5"
format_stamp: "Formatted at 2026-08-09 05:35:23 on dist-test-slave-3kk6"
server_key: "4b9451d6c81ecdae289d3539e7ca3daf"
server_key_iv: "8cc8c4f4af30f08055e86b17e92aefeb"
server_key_version: "encryptionkey@0"
I20260809 05:35:23.862571 6932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:35:23.866300 6955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:23.867826 6932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:23.868034 6932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "8769e6a577044422ba2bfbbba5a9b4f5"
format_stamp: "Formatted at 2026-08-09 05:35:23 on dist-test-slave-3kk6"
server_key: "4b9451d6c81ecdae289d3539e7ca3daf"
server_key_iv: "8cc8c4f4af30f08055e86b17e92aefeb"
server_key_version: "encryptionkey@0"
I20260809 05:35:23.868270 6932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:23.917943 6932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:23.918964 6932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:23.919341 6932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:23.938995 6932 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:39815
I20260809 05:35:23.939034 7029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:39815 every 8 connection(s)
I20260809 05:35:23.940452 6932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:23.944350 7030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:23.945947 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 6932
I20260809 05:35:23.946163 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:23.947114 31271 external_mini_cluster.cc:1428] Setting key 61be7bfce234e78402b71f13cde01785
I20260809 05:35:23.959729 7030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5: Bootstrap starting.
I20260809 05:35:23.961808 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:23.949153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.30.137.254:39815)
0809 05:35:23.950246 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:23.950256 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:23.950298 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:35:23.951937 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:23.951963 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:23.952421 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:23.952741 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:23.953333 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.954627 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.956595 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:23.957786 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:23.957942 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:23.959357 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:23.959366 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:23.959381 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:23.960000 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:23.960102 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:23.960131 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:23.960220 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:23.960475 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 05:35:23.960830 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":433,"threads_started":1}
I20260809 05:35:23.963438 7030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:23.964613 7030 log.cc:824] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:23.965971 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:35:23.967724 7030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5: No bootstrap required, opened a new log
I20260809 05:35:23.975834 7030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39815 } }
I20260809 05:35:23.976246 7030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:23.976377 7030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8769e6a577044422ba2bfbbba5a9b4f5, State: Initialized, Role: FOLLOWER
I20260809 05:35:23.976954 7030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39815 } }
I20260809 05:35:23.977207 7030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:23.977368 7030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:23.977546 7030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:23.980181 7030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39815 } }
I20260809 05:35:23.980762 7030 leader_election.cc:302] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8769e6a577044422ba2bfbbba5a9b4f5; no voters:
I20260809 05:35:23.981406 7030 leader_election.cc:288] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:23.981564 7035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:23.982326 7035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [term 1 LEADER]: Becoming Leader. State: Replica: 8769e6a577044422ba2bfbbba5a9b4f5, State: Running, Role: LEADER
I20260809 05:35:23.983013 7035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39815 } }
I20260809 05:35:23.983809 7030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:23.985898 7036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39815 } } }
I20260809 05:35:23.985924 7038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8769e6a577044422ba2bfbbba5a9b4f5. Latest consensus state: current_term: 1 leader_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8769e6a577044422ba2bfbbba5a9b4f5" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39815 } } }
I20260809 05:35:23.986413 7036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:23.986478 7038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:23.987512 7044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:23.995397 7044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:24.004788 7044 catalog_manager.cc:1347] Generated new cluster ID: 8cdfa4f9282d43a48ea8b8fbc69c6528
I20260809 05:35:24.004926 7044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:24.024483 7044 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:24.026417 7044 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:24.042121 7044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8769e6a577044422ba2bfbbba5a9b4f5: Generated new TSK 0
I20260809 05:35:24.043396 7044 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 05:35:24.081616 7077 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 05:35:24.081770 7077 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 05:35:24.082993 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:24.062125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.30.137.254:39815)
0809 05:35:24.062370 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:24.062375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:24.062774 (+ 399us) server_negotiation.cc:407] Connection header received
0809 05:35:24.063042 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:24.063047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:24.063126 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:24.063278 (+ 152us) server_negotiation.cc:226] Negotiated authn=JWT
0809 05:35:24.063873 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.064592 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.066794 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.067218 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.067325 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:24.067977 (+ 652us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 05:35:24.082332 (+ 14355us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 05:35:24.082747 (+ 415us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":99}
W20260809 05:35:24.083155 7033 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46836: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 05:35:24.083467 7074 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:24.061878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:39815 (local address 127.0.0.1:46836)
0809 05:35:24.062615 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:24.062639 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:24.062860 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:24.063327 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:24.063333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:24.063348 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0809 05:35:24.063677 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:24.063690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.064762 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.064768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:24.066564 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:24.066580 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.067421 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.067428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:24.067625 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:24.067675 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 05:35:24.082628 (+ 14953us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 05:35:24.083007 (+ 379us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.137.254:39815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42147/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":387,"threads_started":1}
I20260809 05:35:24.085482 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 6932
I20260809 05:35:24.088162 7021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:24.186712 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 6932
2026-08-09T05:35:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (830 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 05:35:24.249528 31271 mini_oidc.cc:152] Starting JWKS server
I20260809 05:35:24.251607 31271 webserver.cc:460] Webserver started at https://127.0.0.1:36499/ using document root <none> and password file <none>
I20260809 05:35:24.251904 31271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 05:35:24.252570 31271 webserver.cc:460] Webserver started at http://127.0.0.1:44977/ using document root <none> and password file <none>
2026-08-09T05:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:24Z Disabled control of system clock
I20260809 05:35:24.288590 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42985
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:42985
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36499/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 05:35:24.628175 7090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:24.628541 7090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:24.628713 7090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:24.636642 7090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:24.636811 7090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:24.636899 7090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:24.637012 7090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:24.637094 7090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 05:35:24.637184 7090 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 05:35:24.637265 7090 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36499/jwks.json
I20260809 05:35:24.646502 7090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:42985
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36499/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:24.647931 7090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:24.649775 7090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:24.661901 7103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:24.661967 7099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:24.661993 7100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:24.663173 7090 server_base.cc:1034] running on GCE node
I20260809 05:35:24.664043 7090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:24.665460 7090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:24.666757 7090 hybrid_clock.cc:648] HybridClock initialized: now 1786253724666701 us; error 59 us; skew 500 ppm
I20260809 05:35:24.667145 7090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:24.669515 7090 webserver.cc:460] Webserver started at http://127.30.137.254:37239/ using document root <none> and password file <none>
I20260809 05:35:24.670218 7090 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:24.670367 7090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:24.670775 7090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:24.673712 7090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "dab9729f92a64570a325727bc243ab39"
format_stamp: "Formatted at 2026-08-09 05:35:24 on dist-test-slave-3kk6"
server_key: "fb0818e549cb97f0e608fef90eec4137"
server_key_iv: "3c8012362cbc79316410aa0b5a3a2294"
server_key_version: "encryptionkey@0"
I20260809 05:35:24.674547 7090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "dab9729f92a64570a325727bc243ab39"
format_stamp: "Formatted at 2026-08-09 05:35:24 on dist-test-slave-3kk6"
server_key: "fb0818e549cb97f0e608fef90eec4137"
server_key_iv: "3c8012362cbc79316410aa0b5a3a2294"
server_key_version: "encryptionkey@0"
I20260809 05:35:24.679734 7090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 05:35:24.683290 7113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:24.684628 7090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:24.684832 7090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "dab9729f92a64570a325727bc243ab39"
format_stamp: "Formatted at 2026-08-09 05:35:24 on dist-test-slave-3kk6"
server_key: "fb0818e549cb97f0e608fef90eec4137"
server_key_iv: "3c8012362cbc79316410aa0b5a3a2294"
server_key_version: "encryptionkey@0"
I20260809 05:35:24.685065 7090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:24.710021 7090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:24.711035 7090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:24.711387 7090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:24.730568 7090 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:42985
I20260809 05:35:24.730587 7197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:42985 every 8 connection(s)
I20260809 05:35:24.731966 7090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:24.735641 7198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:24.742255 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 7090
I20260809 05:35:24.742491 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:24.743188 31271 external_mini_cluster.cc:1428] Setting key d12232cf63e1bddacc22d4d324c66b1d
I20260809 05:35:24.746497 7198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39: Bootstrap starting.
I20260809 05:35:24.750751 7198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:24.752164 7198 log.cc:824] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:24.755434 7198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39: No bootstrap required, opened a new log
I20260809 05:35:24.759694 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:24.745160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.30.137.254:42985)
0809 05:35:24.746270 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:24.746286 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:24.746340 (+ 54us) server_negotiation.cc:407] Connection header received
0809 05:35:24.748925 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:24.748966 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:24.749556 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:24.750009 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:24.750551 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.752672 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.754412 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.755478 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.755658 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:24.756765 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:24.756778 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:24.756796 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:24.757545 (+ 749us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:24.757646 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:24.757676 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:24.757797 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:24.758007 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 05:35:24.758363 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":330,"threads_started":1}
I20260809 05:35:24.763744 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:35:24.765149 7198 raft_consensus.cc:348] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab9729f92a64570a325727bc243ab39" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42985 } }
I20260809 05:35:24.765539 7198 raft_consensus.cc:374] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:24.765658 7198 raft_consensus.cc:729] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dab9729f92a64570a325727bc243ab39, State: Initialized, Role: FOLLOWER
I20260809 05:35:24.766299 7198 consensus_queue.cc:260] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab9729f92a64570a325727bc243ab39" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42985 } }
I20260809 05:35:24.766561 7198 raft_consensus.cc:388] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:24.766743 7198 raft_consensus.cc:482] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:24.766945 7198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:24.769174 7198 raft_consensus.cc:504] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab9729f92a64570a325727bc243ab39" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42985 } }
I20260809 05:35:24.769665 7198 leader_election.cc:302] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dab9729f92a64570a325727bc243ab39; no voters:
I20260809 05:35:24.770299 7198 leader_election.cc:288] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:24.770473 7205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:24.771255 7205 raft_consensus.cc:686] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [term 1 LEADER]: Becoming Leader. State: Replica: dab9729f92a64570a325727bc243ab39, State: Running, Role: LEADER
I20260809 05:35:24.771844 7205 consensus_queue.cc:237] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab9729f92a64570a325727bc243ab39" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42985 } }
I20260809 05:35:24.772464 7198 sys_catalog.cc:564] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:24.775039 7209 sys_catalog.cc:455] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dab9729f92a64570a325727bc243ab39. Latest consensus state: current_term: 1 leader_uuid: "dab9729f92a64570a325727bc243ab39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab9729f92a64570a325727bc243ab39" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42985 } } }
I20260809 05:35:24.775501 7209 sys_catalog.cc:458] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:24.777912 7219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:24.777792 7206 sys_catalog.cc:455] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dab9729f92a64570a325727bc243ab39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dab9729f92a64570a325727bc243ab39" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42985 } } }
I20260809 05:35:24.778141 7206 sys_catalog.cc:458] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:24.783612 7219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:24.793411 7219 catalog_manager.cc:1347] Generated new cluster ID: d03d88b03fd1478e9a111d23e4eb45e2
I20260809 05:35:24.793547 7219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:24.803304 7219 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:24.804575 7219 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:24.812538 7219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dab9729f92a64570a325727bc243ab39: Generated new TSK 0
I20260809 05:35:24.813432 7219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:35:24.844170 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:24.835448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.30.137.254:42985)
0809 05:35:24.835699 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:24.835706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:24.836105 (+ 399us) server_negotiation.cc:407] Connection header received
0809 05:35:24.836343 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:24.836347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:24.836431 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:24.836556 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0809 05:35:24.837140 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.837843 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.839922 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.840364 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.840461 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:24.841091 (+ 630us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 05:35:24.843538 (+ 2447us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 05:35:24.843863 (+ 325us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57400: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 05:35:24.844307 7203 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57400: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 05:35:24.846303 7247 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:24.835334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42985 (local address 127.0.0.1:57400)
0809 05:35:24.835989 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:24.836010 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:24.836172 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:24.836612 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:24.836617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:24.836633 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0809 05:35:24.836978 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:24.836992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.837997 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.838004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:24.839775 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:24.839792 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:24.840571 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:24.840580 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:24.840793 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:24.840843 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 05:35:24.843762 (+ 2919us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 05:35:24.845838 (+ 2076us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.137.254:42985: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36499/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":1729,"thread_start_us":313,"threads_started":1}
I20260809 05:35:24.848443 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 7090
I20260809 05:35:24.849622 7184 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 05:35:24.859431 7249 webserver.cc:533] Webserver: sslize failed:
E20260809 05:35:24.859571 7249 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 05:35:24.943744 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 7090
2026-08-09T05:35:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (758 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T05:35:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:25Z Disabled control of system clock
I20260809 05:35:25.044579 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43587
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43587
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/testchainca.cert with env {}
W20260809 05:35:25.385622 7257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:25.385947 7257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:25.386036 7257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:25.393282 7257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:25.393405 7257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:25.393445 7257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:25.393498 7257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:25.393537 7257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 05:35:25.393603 7257 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 05:35:25.393649 7257 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 05:35:25.403926 7257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:25.405243 7257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:25.406456 7257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:25.419071 7267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:25.419158 7270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:25.419114 7273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:25.420362 7257 server_base.cc:1034] running on GCE node
I20260809 05:35:25.421159 7257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:25.422616 7257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:25.423898 7257 hybrid_clock.cc:648] HybridClock initialized: now 1786253725423855 us; error 32 us; skew 500 ppm
I20260809 05:35:25.424265 7257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:35:25.426751 7257 webserver.cc:460] Webserver started at http://127.30.137.254:36987/ using document root <none> and password file <none>
I20260809 05:35:25.427482 7257 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:25.427609 7257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:25.427951 7257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:25.431077 7257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "b731dababe624b1ab34cbdfb69760784"
format_stamp: "Formatted at 2026-08-09 05:35:25 on dist-test-slave-3kk6"
server_key: "e2c37d7bf63de117a78261c41576cc76"
server_key_iv: "90b6477e212c54f5f337b564289b4e19"
server_key_version: "encryptionkey@0"
I20260809 05:35:25.431933 7257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "b731dababe624b1ab34cbdfb69760784"
format_stamp: "Formatted at 2026-08-09 05:35:25 on dist-test-slave-3kk6"
server_key: "e2c37d7bf63de117a78261c41576cc76"
server_key_iv: "90b6477e212c54f5f337b564289b4e19"
server_key_version: "encryptionkey@0"
I20260809 05:35:25.437142 7257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:35:25.440730 7281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:25.442070 7257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:35:25.442276 7257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "b731dababe624b1ab34cbdfb69760784"
format_stamp: "Formatted at 2026-08-09 05:35:25 on dist-test-slave-3kk6"
server_key: "e2c37d7bf63de117a78261c41576cc76"
server_key_iv: "90b6477e212c54f5f337b564289b4e19"
server_key_version: "encryptionkey@0"
I20260809 05:35:25.442533 7257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:25.465766 7257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:25.466769 7257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:25.467088 7257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:25.486364 7257 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:43587
I20260809 05:35:25.486392 7372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:43587 every 8 connection(s)
I20260809 05:35:25.487913 7257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:25.489682 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 7257
I20260809 05:35:25.489887 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:25.490607 31271 external_mini_cluster.cc:1428] Setting key c8e95751dc17cb3d8da84bee3f5ce65c
I20260809 05:35:25.492381 7374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:25.505064 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:25.492739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.30.137.254:43587)
0809 05:35:25.493747 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:25.493758 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:25.493803 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:35:25.495478 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:25.495502 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:25.495984 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:25.496352 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:25.496998 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:25.498378 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:25.500295 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:25.500985 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:25.501127 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:25.502472 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:25.502480 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:25.502493 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:35:25.503162 (+ 669us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:25.503261 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:35:25.503284 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:25.503443 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:25.503691 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 05:35:25.504042 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":373,"threads_started":1}
I20260809 05:35:25.506778 7374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784: Bootstrap starting.
I20260809 05:35:25.509033 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:35:25.510268 7374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:25.511554 7374 log.cc:824] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:25.514587 7374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784: No bootstrap required, opened a new log
I20260809 05:35:25.522848 7374 raft_consensus.cc:348] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731dababe624b1ab34cbdfb69760784" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43587 } }
I20260809 05:35:25.523217 7374 raft_consensus.cc:374] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:25.523370 7374 raft_consensus.cc:729] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b731dababe624b1ab34cbdfb69760784, State: Initialized, Role: FOLLOWER
I20260809 05:35:25.523926 7374 consensus_queue.cc:260] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731dababe624b1ab34cbdfb69760784" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43587 } }
I20260809 05:35:25.524160 7374 raft_consensus.cc:388] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:25.524372 7374 raft_consensus.cc:482] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:25.524684 7374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:25.527439 7374 raft_consensus.cc:504] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731dababe624b1ab34cbdfb69760784" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43587 } }
I20260809 05:35:25.527959 7374 leader_election.cc:302] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b731dababe624b1ab34cbdfb69760784; no voters:
I20260809 05:35:25.528574 7374 leader_election.cc:288] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:25.528795 7381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:25.529533 7381 raft_consensus.cc:686] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [term 1 LEADER]: Becoming Leader. State: Replica: b731dababe624b1ab34cbdfb69760784, State: Running, Role: LEADER
I20260809 05:35:25.530056 7381 consensus_queue.cc:237] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731dababe624b1ab34cbdfb69760784" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43587 } }
I20260809 05:35:25.530866 7374 sys_catalog.cc:564] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:25.533002 7382 sys_catalog.cc:455] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b731dababe624b1ab34cbdfb69760784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731dababe624b1ab34cbdfb69760784" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43587 } } }
I20260809 05:35:25.533479 7382 sys_catalog.cc:458] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:25.533187 7383 sys_catalog.cc:455] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b731dababe624b1ab34cbdfb69760784. Latest consensus state: current_term: 1 leader_uuid: "b731dababe624b1ab34cbdfb69760784" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b731dababe624b1ab34cbdfb69760784" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43587 } } }
I20260809 05:35:25.533675 7383 sys_catalog.cc:458] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:25.535286 7389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:25.540576 7389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:25.550115 7389 catalog_manager.cc:1347] Generated new cluster ID: 46ad50912f5941b480e9210659d4f620
I20260809 05:35:25.550253 7389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:25.565989 7389 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:25.567534 7389 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:25.573685 7389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b731dababe624b1ab34cbdfb69760784: Generated new TSK 0
I20260809 05:35:25.574546 7389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:35:26.150874 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:25.598092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53808 (local address 127.30.137.254:43587)
0809 05:35:25.598300 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:25.598307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:25.598869 (+ 562us) server_negotiation.cc:407] Connection header received
0809 05:35:25.599125 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:25.599132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:25.599240 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:25.599428 (+ 188us) server_negotiation.cc:226] Negotiated authn=JWT
0809 05:35:25.599886 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:25.600998 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:25.602439 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:25.603180 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:25.603320 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:25.603796 (+ 476us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 05:35:26.150209 (+546413us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 05:35:26.150561 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53808: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 05:35:26.151031 7378 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53808: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 05:35:26.151165 7421 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:25.598024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43587 (local address 127.0.0.1:53808)
0809 05:35:25.598760 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:25.598777 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:25.598927 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:25.599456 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:25.599460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:25.599470 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 05:35:25.599717 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:25.599725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:25.601163 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:25.601167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:25.602285 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:25.602295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:25.603350 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:25.603354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:25.603474 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:25.603506 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 05:35:26.150520 (+547014us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 05:35:26.150830 (+ 310us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.137.254:43587: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":367,"threads_started":1}
I20260809 05:35:26.152819 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 7257
I20260809 05:35:26.153947 7367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:26.257071 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 7257
2026-08-09T05:35:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1294 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (428 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (435 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:27 dist-test-slave-3kk6 krb5kdc[7445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:27 dist-test-slave-3kk6 krb5kdc[7445](info): set up 2 sockets
Aug 09 05:35:27 dist-test-slave-3kk6 krb5kdc[7445](info): commencing operation
krb5kdc: starting...
W20260809 05:35:28.461956 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:28 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:28.731078 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35829
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:29.069525 7467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:29.069844 7467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:29.069939 7467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:29.077139 7467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:29.077270 7467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:29.077320 7467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:29.077381 7467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:29.077425 7467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:29.087466 7467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:29.088815 7467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:29.090129 7467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:29.102375 7472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:29.102397 7475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:29.102510 7473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:29.103509 7467 server_base.cc:1034] running on GCE node
I20260809 05:35:29.104285 7467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:29.106494 7467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:29.107748 7467 hybrid_clock.cc:648] HybridClock initialized: now 1786253729107621 us; error 117 us; skew 500 ppm
I20260809 05:35:29.108242 7467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:29 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253729, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:29.115100 7467 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:29.116724 7467 webserver.cc:460] Webserver started at http://127.30.137.254:33873/ using document root <none> and password file <none>
I20260809 05:35:29.117484 7467 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:29.117666 7467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:29.118018 7467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:29.121284 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "52b87c1420fd4fc3a378cadc2c668155"
format_stamp: "Formatted at 2026-08-09 05:35:29 on dist-test-slave-3kk6"
server_key: "3828829ce5d2df712a0b4d291f02c21e"
server_key_iv: "961d60231f45bb2fdf936c01cafa3bff"
server_key_version: "encryptionkey@0"
I20260809 05:35:29.122123 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "52b87c1420fd4fc3a378cadc2c668155"
format_stamp: "Formatted at 2026-08-09 05:35:29 on dist-test-slave-3kk6"
server_key: "3828829ce5d2df712a0b4d291f02c21e"
server_key_iv: "961d60231f45bb2fdf936c01cafa3bff"
server_key_version: "encryptionkey@0"
I20260809 05:35:29.127395 7467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 05:35:29.130808 7488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:29.132146 7467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:35:29.132333 7467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "52b87c1420fd4fc3a378cadc2c668155"
format_stamp: "Formatted at 2026-08-09 05:35:29 on dist-test-slave-3kk6"
server_key: "3828829ce5d2df712a0b4d291f02c21e"
server_key_iv: "961d60231f45bb2fdf936c01cafa3bff"
server_key_version: "encryptionkey@0"
I20260809 05:35:29.132544 7467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:29.147420 7467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:29.152105 7467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:29.152431 7467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:29.171132 7467 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35829
I20260809 05:35:29.171170 7567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35829 every 8 connection(s)
I20260809 05:35:29.172644 7467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:29.176750 7570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:29.176810 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 7467
I20260809 05:35:29.177026 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:29.177881 31271 external_mini_cluster.cc:1428] Setting key 1202a8b6cff8f55b002167033528e834
I20260809 05:35:29.191612 7570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155: Bootstrap starting.
Aug 09 05:35:29 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:29.195639 7570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:29.197062 7570 log.cc:824] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:29.200656 7570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155: No bootstrap required, opened a new log
I20260809 05:35:29.203982 7576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:29.180146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.30.137.254:35829)
0809 05:35:29.180994 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:29.181006 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:29.181056 (+ 50us) server_negotiation.cc:407] Connection header received
0809 05:35:29.183401 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:29.183426 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:29.183844 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:29.184222 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:29.186207 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:29.187628 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:29.189562 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:29.190255 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:29.190391 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:29.195323 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:29.195337 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:29.195359 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:29.195412 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:29.199279 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:29.199987 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:29.199999 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:29.200011 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:29.200226 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:29.200598 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:29.200604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:29.200611 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:29.201414 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:29.201731 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:29.202071 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 05:35:29.202508 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":407,"threads_started":1}
I20260809 05:35:29.209857 7570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b87c1420fd4fc3a378cadc2c668155" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35829 } }
I20260809 05:35:29.210327 7570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:29.210410 7570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b87c1420fd4fc3a378cadc2c668155, State: Initialized, Role: FOLLOWER
I20260809 05:35:29.210963 7570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b87c1420fd4fc3a378cadc2c668155" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35829 } }
I20260809 05:35:29.211175 7570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:29.211278 7570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:29.211453 7570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:29.213969 7570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b87c1420fd4fc3a378cadc2c668155" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35829 } }
I20260809 05:35:29.214525 7570 leader_election.cc:302] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b87c1420fd4fc3a378cadc2c668155; no voters:
I20260809 05:35:29.215167 7570 leader_election.cc:288] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:29.215304 7579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:29.216585 7579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [term 1 LEADER]: Becoming Leader. State: Replica: 52b87c1420fd4fc3a378cadc2c668155, State: Running, Role: LEADER
I20260809 05:35:29.217207 7579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b87c1420fd4fc3a378cadc2c668155" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35829 } }
I20260809 05:35:29.217424 7570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:29.220220 7581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52b87c1420fd4fc3a378cadc2c668155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b87c1420fd4fc3a378cadc2c668155" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35829 } } }
I20260809 05:35:29.220701 7581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:29.220232 7582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b87c1420fd4fc3a378cadc2c668155. Latest consensus state: current_term: 1 leader_uuid: "52b87c1420fd4fc3a378cadc2c668155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b87c1420fd4fc3a378cadc2c668155" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35829 } } }
I20260809 05:35:29.221272 7582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:29.223917 7587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:29.229158 7587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:29.237534 7587 catalog_manager.cc:1347] Generated new cluster ID: 4fadbb41ec4e4f0abe0ada5006c2aea0
I20260809 05:35:29.237654 7587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:29.251452 7587 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:29.252799 7587 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:29.262405 7587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52b87c1420fd4fc3a378cadc2c668155: Generated new TSK 0
I20260809 05:35:29.263549 7587 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:35:29.330492 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35829
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:29.684876 7606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:29.685241 7606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:29.685446 7606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:29.693059 7606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:29.693253 7606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:29.693451 7606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:29.704280 7606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:29.705693 7606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:29.707211 7606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:29.721517 7614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:29.721642 7617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:29.721557 7613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:29.722879 7606 server_base.cc:1034] running on GCE node
I20260809 05:35:29.723644 7606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:29.724761 7606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:29.725976 7606 hybrid_clock.cc:648] HybridClock initialized: now 1786253729725937 us; error 42 us; skew 500 ppm
I20260809 05:35:29.726331 7606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:29 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253729, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:29.729782 7606 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:35:29.731551 7606 webserver.cc:460] Webserver started at http://127.30.137.193:40307/ using document root <none> and password file <none>
I20260809 05:35:29.732302 7606 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:29.732447 7606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:29.732800 7606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:29.736140 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "b1bb92d6cf404f17aaf2157ac86728ea"
format_stamp: "Formatted at 2026-08-09 05:35:29 on dist-test-slave-3kk6"
server_key: "a2d386d0640f761a2cddab102913f7ca"
server_key_iv: "25e80b46ffdbfcb81b9340ce6a727dfc"
server_key_version: "encryptionkey@0"
I20260809 05:35:29.737068 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "b1bb92d6cf404f17aaf2157ac86728ea"
format_stamp: "Formatted at 2026-08-09 05:35:29 on dist-test-slave-3kk6"
server_key: "a2d386d0640f761a2cddab102913f7ca"
server_key_iv: "25e80b46ffdbfcb81b9340ce6a727dfc"
server_key_version: "encryptionkey@0"
I20260809 05:35:29.742784 7606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 05:35:29.746470 7625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:29.748070 7606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 05:35:29.748263 7606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "b1bb92d6cf404f17aaf2157ac86728ea"
format_stamp: "Formatted at 2026-08-09 05:35:29 on dist-test-slave-3kk6"
server_key: "a2d386d0640f761a2cddab102913f7ca"
server_key_iv: "25e80b46ffdbfcb81b9340ce6a727dfc"
server_key_version: "encryptionkey@0"
I20260809 05:35:29.748498 7606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:29.790529 7606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:29.795025 7606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:29.795388 7606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:29.796443 7606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:29.797935 7606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:29.798048 7606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:29.798206 7606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:29.798283 7606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:29.833803 7606 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:37077
I20260809 05:35:29.833868 7793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:37077 every 8 connection(s)
I20260809 05:35:29.835193 7606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:29.838871 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 7606
I20260809 05:35:29.839115 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:29.840054 31271 external_mini_cluster.cc:1428] Setting key 88f9acfa4e255c3006f7813a0339dde0
Aug 09 05:35:29 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253729, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:29.860558 7798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:29.838059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:59707 (local address 127.30.137.254:35829)
0809 05:35:29.838675 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:29.838683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:29.839842 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 05:35:29.842174 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:29.842187 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:29.842266 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:29.842362 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:29.844613 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:29.845351 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:29.847857 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:29.848250 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:29.848351 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:29.852915 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:29.852925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:29.852931 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:29.852981 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:29.856894 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:29.857687 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:29.857696 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:29.857702 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:29.857857 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:29.858385 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:29.858393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:29.858399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:29.858876 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:29.859031 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:29.860092 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0809 05:35:29.860289 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":293,"threads_started":1}
I20260809 05:35:29.862222 7799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:29.838766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35829 (local address 127.30.137.193:59707)
0809 05:35:29.839632 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:29.839690 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:29.841869 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:29.842674 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:29.842688 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:29.843539 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:29.844409 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:29.844439 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:29.845519 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:29.845528 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:29.847713 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:29.847731 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:29.848484 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:29.848493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:29.848833 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:29.849660 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:29.849694 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:29.852619 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:29.857062 (+ 4443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:29.857070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:29.857084 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:29.857540 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:29.858002 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:29.858006 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:29.858009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:29.858249 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:29.859121 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:29.859131 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:29.859625 (+ 494us) client_negotiation.cc:769] Sending connection context
0809 05:35:29.859957 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 05:35:29.860635 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":366,"threads_started":1}
I20260809 05:35:29.863996 7796 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35829
I20260809 05:35:29.864348 7796 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:29.865373 7796 heartbeater.cc:511] Master 127.30.137.254:35829 requested a full tablet report, sending...
I20260809 05:35:29.867849 7512 ts_manager.cc:194] Registered new tserver with Master: b1bb92d6cf404f17aaf2157ac86728ea (127.30.137.193:37077)
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:35:29.869439 7512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:59707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:35:29.903031 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35829
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:30.262557 7806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:30.262995 7806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:30.263185 7806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:30.270792 7806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:30.270903 7806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:30.271082 7806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:30.281067 7806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:30.282441 7806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:30.283898 7806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:30.297801 7817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:30.297839 7819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:30.297793 7815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:30.298729 7806 server_base.cc:1034] running on GCE node
I20260809 05:35:30.299506 7806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:30.300565 7806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:30.301824 7806 hybrid_clock.cc:648] HybridClock initialized: now 1786253730301785 us; error 60 us; skew 500 ppm
I20260809 05:35:30.302183 7806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:30 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253730, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:30.305648 7806 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:35:30.307330 7806 webserver.cc:460] Webserver started at http://127.30.137.194:46693/ using document root <none> and password file <none>
I20260809 05:35:30.308071 7806 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:30.308231 7806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:30.308562 7806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:30.311837 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "c712c3d1ac1f44038f182d6fd4c856b6"
format_stamp: "Formatted at 2026-08-09 05:35:30 on dist-test-slave-3kk6"
server_key: "1076b04807198bce700f5af0a2497263"
server_key_iv: "6a839bf296c54ac5b6c0a4710990c3f4"
server_key_version: "encryptionkey@0"
I20260809 05:35:30.312681 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "c712c3d1ac1f44038f182d6fd4c856b6"
format_stamp: "Formatted at 2026-08-09 05:35:30 on dist-test-slave-3kk6"
server_key: "1076b04807198bce700f5af0a2497263"
server_key_iv: "6a839bf296c54ac5b6c0a4710990c3f4"
server_key_version: "encryptionkey@0"
I20260809 05:35:30.318378 7806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:30.322139 7832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:30.323616 7806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:30.323817 7806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "c712c3d1ac1f44038f182d6fd4c856b6"
format_stamp: "Formatted at 2026-08-09 05:35:30 on dist-test-slave-3kk6"
server_key: "1076b04807198bce700f5af0a2497263"
server_key_iv: "6a839bf296c54ac5b6c0a4710990c3f4"
server_key_version: "encryptionkey@0"
I20260809 05:35:30.324064 7806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:30.346228 7806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:30.350611 7806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:30.350900 7806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:30.351994 7806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:30.353513 7806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:30.353677 7806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 05:35:30.353866 7806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:30.353946 7806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:30.407855 7806 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:33375
I20260809 05:35:30.407953 7998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:33375 every 8 connection(s)
I20260809 05:35:30.409816 7806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:30.415436 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 7806
I20260809 05:35:30.415643 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:30.416353 31271 external_mini_cluster.cc:1428] Setting key 3a5c9a622d33a1e45a2570da88635849
Aug 09 05:35:30 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253730, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:30.449539 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:30.414386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:37131 (local address 127.30.137.254:35829)
0809 05:35:30.414936 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:30.414943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:30.418283 (+ 3340us) server_negotiation.cc:407] Connection header received
0809 05:35:30.430902 (+ 12619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:30.430912 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:30.431023 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:30.431156 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:30.433905 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.435022 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.436649 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.437171 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.437272 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:30.441818 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:30.441828 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:30.441835 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:30.441886 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:30.445803 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:30.446638 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:30.446647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:30.446676 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:30.446862 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:30.447312 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:30.447320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:30.447326 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:30.447767 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:30.447928 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:30.448707 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 05:35:30.449007 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":278,"threads_started":1}
I20260809 05:35:30.450181 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:30.413756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35829 (local address 127.30.137.194:37131)
0809 05:35:30.418088 (+ 4332us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:30.418150 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:30.430426 (+ 12276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:30.431694 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:30.431711 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:30.432654 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:30.433571 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:30.433603 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.435200 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.435204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:30.436511 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:30.436524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.437419 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.437429 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:30.437799 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:30.438601 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:30.438637 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:30.441522 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:30.445984 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:30.445992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:30.446007 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:30.446495 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:30.446980 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:30.446984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:30.446986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:30.447187 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:30.447910 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:30.447916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:30.448268 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 05:35:30.448543 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 05:35:30.448970 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3981,"thread_start_us":951,"threads_started":1}
I20260809 05:35:30.451889 7999 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35829
I20260809 05:35:30.452265 7999 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
I20260809 05:35:30.453181 7999 heartbeater.cc:511] Master 127.30.137.254:35829 requested a full tablet report, sending...
I20260809 05:35:30.455019 7512 ts_manager.cc:194] Registered new tserver with Master: c712c3d1ac1f44038f182d6fd4c856b6 (127.30.137.194:33375)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
I20260809 05:35:30.455950 7512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:37131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:35:30.486008 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35829
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:30.840317 8006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:30.840639 8006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:30.840806 8006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:30.848239 8006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:30.848402 8006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:30.848596 8006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:30.858738 8006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:30.860230 8006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:30.861572 8006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 05:35:30.872810 7796 heartbeater.cc:503] Master 127.30.137.254:35829 was elected leader, sending a full tablet report...
W20260809 05:35:30.877916 8013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:30.877965 8014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:30.877876 8016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:30.878991 8006 server_base.cc:1034] running on GCE node
I20260809 05:35:30.879675 8006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:30.880662 8006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:30.881888 8006 hybrid_clock.cc:648] HybridClock initialized: now 1786253730881843 us; error 48 us; skew 500 ppm
I20260809 05:35:30.882220 8006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:30 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253730, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:30.885584 8006 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:35:30.887187 8006 webserver.cc:460] Webserver started at http://127.30.137.195:46399/ using document root <none> and password file <none>
I20260809 05:35:30.887904 8006 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:30.888013 8006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:30.888293 8006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:30.891472 8006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "9680f300ea22446aa91d4654d71b00ed"
format_stamp: "Formatted at 2026-08-09 05:35:30 on dist-test-slave-3kk6"
server_key: "4e2a0db1f8d57ab06133eea60654447e"
server_key_iv: "8411ecb61c843302dc799603a62558c3"
server_key_version: "encryptionkey@0"
I20260809 05:35:30.892323 8006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "9680f300ea22446aa91d4654d71b00ed"
format_stamp: "Formatted at 2026-08-09 05:35:30 on dist-test-slave-3kk6"
server_key: "4e2a0db1f8d57ab06133eea60654447e"
server_key_iv: "8411ecb61c843302dc799603a62558c3"
server_key_version: "encryptionkey@0"
I20260809 05:35:30.897748 8006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:35:30.901501 8025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:30.902971 8006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:30.903119 8006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "9680f300ea22446aa91d4654d71b00ed"
format_stamp: "Formatted at 2026-08-09 05:35:30 on dist-test-slave-3kk6"
server_key: "4e2a0db1f8d57ab06133eea60654447e"
server_key_iv: "8411ecb61c843302dc799603a62558c3"
server_key_version: "encryptionkey@0"
I20260809 05:35:30.903326 8006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:30.917711 8006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:30.921995 8006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:30.922289 8006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:30.923338 8006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:30.924753 8006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:30.924827 8006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:30.924949 8006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:30.925012 8006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:30.959535 8006 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:45337
I20260809 05:35:30.959586 8195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:45337 every 8 connection(s)
I20260809 05:35:30.961018 8006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:30.963193 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 8006
I20260809 05:35:30.963372 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:30.963909 31271 external_mini_cluster.cc:1428] Setting key 6400279bd2ff509a4b19c48c2c7e6e54
Aug 09 05:35:30 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253730, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:30.982039 8200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:30.963868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:47605 (local address 127.30.137.254:35829)
0809 05:35:30.966471 (+ 2603us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:30.966476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:30.966492 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:35:30.968026 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:30.968031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:30.968108 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:30.968200 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:30.969667 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.970452 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.972072 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.972479 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.972579 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:30.975941 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:30.975946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:30.975949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:30.975983 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:30.979023 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:30.980002 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:30.980007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:30.980009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:30.980135 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:30.980533 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:30.980536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:30.980538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:30.980796 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:30.980956 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:30.981581 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 05:35:30.981760 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2508,"thread_start_us":2363,"threads_started":1}
I20260809 05:35:30.983072 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:30.964557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35829 (local address 127.30.137.195:47605)
0809 05:35:30.965602 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:30.965663 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:30.967786 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:30.968428 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:30.968435 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:30.968958 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:30.969525 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:30.969542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.970587 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.970591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:30.971936 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:30.971950 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:30.972663 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:30.972668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:30.972859 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:30.973486 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:30.973507 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:30.975753 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:30.979170 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:30.979178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:30.979191 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:30.979868 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:30.980235 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:30.980239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:30.980241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:30.980437 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:30.980899 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:30.980905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:30.981239 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 05:35:30.981476 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 05:35:30.981876 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":421,"threads_started":1}
I20260809 05:35:30.984578 8197 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35829
I20260809 05:35:30.984894 8197 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:30.985589 8197 heartbeater.cc:511] Master 127.30.137.254:35829 requested a full tablet report, sending...
I20260809 05:35:30.987049 7512 ts_manager.cc:194] Registered new tserver with Master: 9680f300ea22446aa91d4654d71b00ed (127.30.137.195:45337)
I20260809 05:35:30.987864 7512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:47605
I20260809 05:35:30.992120 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/test-admin.keytab.
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.024011 31271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 05:35:31.029798 8209 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:31.026860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.30.137.193:37077)
0809 05:35:31.027528 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.027538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.027571 (+ 33us) server_negotiation.cc:407] Connection header received
0809 05:35:31.027779 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.027787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.028026 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.028198 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.029305 (+ 1107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47902: BlockingRecv error: recv got EOF from 127.0.0.1:47902 (error 108)
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":307,"threads_started":1}
W20260809 05:35:31.029914 31271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc' of type FILE
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.031778 31271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.033196 31271 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:31.049214 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:31.034922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.30.137.193:37077)
0809 05:35:31.035131 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.035137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.035328 (+ 191us) server_negotiation.cc:407] Connection header received
0809 05:35:31.035551 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.035557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.035632 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.035747 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.036963 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.038001 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.039329 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.039917 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.040068 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:31.043748 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:31.043762 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:31.043785 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:31.043818 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:31.046576 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.047378 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.047385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.047388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.047517 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.047840 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.047843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.047845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.048227 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:31.048453 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:31.048735 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 05:35:31.048944 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 05:35:31.055267 8209 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:31.053429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.30.137.193:37077)
0809 05:35:31.053654 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.053660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.053942 (+ 282us) server_negotiation.cc:407] Connection header received
0809 05:35:31.054063 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.054066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.054122 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.054201 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.055042 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47922: BlockingRecv error: recv got EOF from 127.0.0.1:47922 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 05:35:31.055418 31271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc' of type FILE
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.057058 31271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.058369 31271 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:31.073136 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:31.059744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47934 (local address 127.30.137.193:37077)
0809 05:35:31.059962 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.059968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.060155 (+ 187us) server_negotiation.cc:407] Connection header received
0809 05:35:31.060301 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.060308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.060394 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.060519 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.061446 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.062148 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.063385 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.063826 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.063923 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:31.067640 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:31.067645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:31.067649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:31.067682 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:31.070437 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.071186 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.071191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.071200 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.071334 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.071769 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.071773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.071775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.072049 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:31.072320 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:31.072636 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 05:35:31.072869 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 05:35:31.077769 8209 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:31.075915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.30.137.193:37077)
0809 05:35:31.076122 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.076131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.076406 (+ 275us) server_negotiation.cc:407] Connection header received
0809 05:35:31.076599 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.076603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.076673 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.076775 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.077523 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47940: BlockingRecv error: recv got EOF from 127.0.0.1:47940 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 05:35:31.078114 31271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786253675866932-31271-0/krb5kdc/krb5cc' of type FILE
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.079699 31271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:31.081081 31271 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:31.095968 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:31.082440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.30.137.193:37077)
0809 05:35:31.082635 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.082643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.083000 (+ 357us) server_negotiation.cc:407] Connection header received
0809 05:35:31.083199 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.083203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.083261 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.083363 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.084325 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.085073 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.086355 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.086863 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.087002 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:31.090845 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:31.090850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:31.090854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:31.090890 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:31.093582 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.094220 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.094223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.094226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.094358 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.094705 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.094708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.094710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.095058 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:31.095312 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:31.095515 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 05:35:31.095687 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 05:35:31.097549 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 7606
I20260809 05:35:31.109810 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:31.098276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.30.137.193:37077)
0809 05:35:31.098472 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.098481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.098866 (+ 385us) server_negotiation.cc:407] Connection header received
0809 05:35:31.099062 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.099067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.099133 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.099232 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.100215 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.100911 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.102140 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.102552 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.102708 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:31.104774 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:31.104780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:31.104783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:31.104813 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:31.107203 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.107857 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.107864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.107867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.107995 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.108370 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.108376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.108379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.108654 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:31.108954 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:31.109247 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 05:35:31.109470 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:35:31.110410 7788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:31.200315 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 7606
I20260809 05:35:31.223188 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 7806
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:35:31.240185 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:31.224208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.30.137.194:33375)
0809 05:35:31.224801 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.224808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.224834 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:35:31.224896 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.224902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.225103 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.225230 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.226230 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.227313 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.228833 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.229414 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.229560 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:31.233887 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:31.233900 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:31.233914 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:31.233959 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:31.237140 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.237810 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.237818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.237820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.237948 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.238324 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.238327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.238329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.238847 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:31.239078 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:31.239363 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 05:35:31.239651 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":327,"threads_started":1}
I20260809 05:35:31.241541 7993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:31.325831 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 7806
I20260809 05:35:31.350302 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 8006
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[7445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:35:31.367143 8235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:31.351617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.30.137.195:45337)
0809 05:35:31.352349 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:31.352356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:31.352387 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:35:31.352453 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:31.352459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:31.352671 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:31.352811 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:31.353861 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.355044 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.356287 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:31.356888 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:31.356993 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:31.361260 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:31.361276 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:31.361292 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:31.361341 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:31.364524 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.365222 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.365230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.365232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.365368 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:31.365686 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:31.365689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:31.365691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:31.366039 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:31.366271 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:31.366499 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 05:35:31.366802 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":359,"threads_started":1}
I20260809 05:35:31.368506 8188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:31.453173 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 8006
I20260809 05:35:31.474326 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 7467
I20260809 05:35:31.475802 7560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:31.581573 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 7467
2026-08-09T05:35:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4463 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[8241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[8241](info): set up 2 sockets
Aug 09 05:35:31 dist-test-slave-3kk6 krb5kdc[8241](info): commencing operation
krb5kdc: starting...
W20260809 05:35:32.910444 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:33 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab.
Entry for principal oryx/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab.
I20260809 05:35:33.182407 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:46411
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab
--principal=oryx/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:46411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:33.522953 8265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:33.523254 8265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:33.523429 8265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:33.531394 8265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:33.531535 8265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:33.531625 8265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:33.531765 8265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:33.531838 8265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:33.542132 8265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:46411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab
--principal=oryx/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:46411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:33.543615 8265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:33.544991 8265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:33.557165 8274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:33.557252 8276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:33.557217 8272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:33.558329 8265 server_base.cc:1034] running on GCE node
I20260809 05:35:33.559269 8265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:33.560791 8265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:33.562007 8265 hybrid_clock.cc:648] HybridClock initialized: now 1786253733561970 us; error 44 us; skew 500 ppm
I20260809 05:35:33.562357 8265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:33 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:33.565786 8265 init.cc:377] Logged in from keytab as oryx/127.30.137.254@KRBTEST.COM (short username oryx)
I20260809 05:35:33.567605 8265 webserver.cc:460] Webserver started at http://127.30.137.254:46107/ using document root <none> and password file <none>
I20260809 05:35:33.568322 8265 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:33.568481 8265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:33.568780 8265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:33.571851 8265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "5503b5dfd58e4712910fac17b2f1da82"
format_stamp: "Formatted at 2026-08-09 05:35:33 on dist-test-slave-3kk6"
server_key: "ed96f2597bdeb1acf444710e3be645f7"
server_key_iv: "edfb7b8b6039da1bb256b99cf5f28c40"
server_key_version: "encryptionkey@0"
I20260809 05:35:33.572582 8265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "5503b5dfd58e4712910fac17b2f1da82"
format_stamp: "Formatted at 2026-08-09 05:35:33 on dist-test-slave-3kk6"
server_key: "ed96f2597bdeb1acf444710e3be645f7"
server_key_iv: "edfb7b8b6039da1bb256b99cf5f28c40"
server_key_version: "encryptionkey@0"
I20260809 05:35:33.577876 8265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 05:35:33.581740 8288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:33.583263 8265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:33.583470 8265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "5503b5dfd58e4712910fac17b2f1da82"
format_stamp: "Formatted at 2026-08-09 05:35:33 on dist-test-slave-3kk6"
server_key: "ed96f2597bdeb1acf444710e3be645f7"
server_key_iv: "edfb7b8b6039da1bb256b99cf5f28c40"
server_key_version: "encryptionkey@0"
I20260809 05:35:33.583693 8265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:33.599596 8265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:33.603888 8265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:33.604203 8265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:33.623584 8265 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:46411
I20260809 05:35:33.623636 8364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:46411 every 8 connection(s)
I20260809 05:35:33.624980 8265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:33.628548 8365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:33.630119 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 8265
I20260809 05:35:33.630362 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:33.631273 31271 external_mini_cluster.cc:1428] Setting key c7bcd87351f49b86de6e5b2411cc6fdd
I20260809 05:35:33.644038 8365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Bootstrap starting.
Aug 09 05:35:33 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:33.647938 8365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:33.649121 8365 log.cc:824] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:33.652243 8365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: No bootstrap required, opened a new log
I20260809 05:35:33.656236 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:33.633433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.30.137.254:46411)
0809 05:35:33.634248 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:33.634260 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:33.634305 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:35:33.636000 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:33.636023 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:33.636432 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:33.636773 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:33.638737 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:33.640539 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:33.642409 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:33.643139 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:33.643274 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:33.648087 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:33.648100 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:33.648119 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:33.648166 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:33.651749 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:33.652477 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:33.652485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:33.652493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:33.652718 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:33.653101 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:33.653107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:33.653116 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:33.653795 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:33.654139 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:33.654410 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 05:35:33.654919 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":380,"threads_started":1}
I20260809 05:35:33.661163 8365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } }
I20260809 05:35:33.661537 8365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:33.661615 8365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5503b5dfd58e4712910fac17b2f1da82, State: Initialized, Role: FOLLOWER
I20260809 05:35:33.662106 8365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } }
I20260809 05:35:33.662334 8365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:33.662488 8365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:33.662750 8365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:33.665107 8365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } }
I20260809 05:35:33.665628 8365 leader_election.cc:302] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5503b5dfd58e4712910fac17b2f1da82; no voters:
I20260809 05:35:33.666277 8365 leader_election.cc:288] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:33.666399 8372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:33.667256 8372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 LEADER]: Becoming Leader. State: Replica: 5503b5dfd58e4712910fac17b2f1da82, State: Running, Role: LEADER
I20260809 05:35:33.668210 8372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } }
I20260809 05:35:33.668610 8365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:33.672037 8373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } }
I20260809 05:35:33.672446 8373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:33.672726 8374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5503b5dfd58e4712910fac17b2f1da82. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } }
I20260809 05:35:33.673493 8374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:33.675410 8390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:33.680639 8390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:33.690017 8390 catalog_manager.cc:1347] Generated new cluster ID: e3949a3c8a6845f29237bf3225b0be8a
I20260809 05:35:33.690165 8390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:33.702332 8390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:33.703794 8390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:33.713097 8390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Generated new TSK 0
I20260809 05:35:33.714321 8390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab.
Entry for principal oryx/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab.
I20260809 05:35:33.781033 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:46411
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:34.131681 8408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:34.132091 8408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:34.132262 8408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:34.139935 8408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:34.140103 8408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:34.140281 8408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:34.150283 8408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:46411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:34.151806 8408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:34.153184 8408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:34.166957 8424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:34.167078 8417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:34.167049 8418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:34.168295 8408 server_base.cc:1034] running on GCE node
I20260809 05:35:34.168977 8408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:34.169956 8408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:34.171164 8408 hybrid_clock.cc:648] HybridClock initialized: now 1786253734171112 us; error 37 us; skew 500 ppm
I20260809 05:35:34.171486 8408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:34 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253734, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:34.179141 8408 init.cc:377] Logged in from keytab as oryx/127.30.137.193@KRBTEST.COM (short username oryx)
I20260809 05:35:34.180788 8408 webserver.cc:460] Webserver started at http://127.30.137.193:45823/ using document root <none> and password file <none>
I20260809 05:35:34.181504 8408 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:34.181614 8408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:34.181887 8408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:34.185017 8408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "938f3ee41234415bb7818fd76ff7b626"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "8cf454d0eb0089897834a4f22598620f"
server_key_iv: "3d22ff9ab6edb43b71ec028e4466ded6"
server_key_version: "encryptionkey@0"
I20260809 05:35:34.185807 8408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "938f3ee41234415bb7818fd76ff7b626"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "8cf454d0eb0089897834a4f22598620f"
server_key_iv: "3d22ff9ab6edb43b71ec028e4466ded6"
server_key_version: "encryptionkey@0"
I20260809 05:35:34.191430 8408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 05:35:34.194947 8433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:34.196331 8408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:35:34.196470 8408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "938f3ee41234415bb7818fd76ff7b626"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "8cf454d0eb0089897834a4f22598620f"
server_key_iv: "3d22ff9ab6edb43b71ec028e4466ded6"
server_key_version: "encryptionkey@0"
I20260809 05:35:34.196645 8408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:34.221519 8408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:34.225965 8408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:34.226276 8408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:34.227342 8408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:34.228700 8408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:34.228775 8408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:34.228861 8408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:34.228912 8408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:34.264570 8408 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:41035
I20260809 05:35:34.264624 8617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:41035 every 8 connection(s)
I20260809 05:35:34.265949 8408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:34.269927 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 8408
I20260809 05:35:34.270160 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:34.270991 31271 external_mini_cluster.cc:1428] Setting key a6de7efac12aa3a3521e8ed80fb24825
Aug 09 05:35:34 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253734, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:34.291158 8620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:34.268703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:37827 (local address 127.30.137.254:46411)
0809 05:35:34.269149 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:34.269153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:34.270238 (+ 1085us) server_negotiation.cc:407] Connection header received
0809 05:35:34.272430 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:34.272435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:34.272546 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:34.272667 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:34.274757 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.275501 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.278030 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.278426 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.278528 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:34.283445 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:34.283450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:34.283454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:34.283492 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:34.287442 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:34.288391 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:34.288399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:34.288406 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:34.288574 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:34.289004 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:34.289012 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:34.289018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:34.289424 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:34.289582 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:34.290348 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 05:35:34.290625 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":219,"threads_started":1}
I20260809 05:35:34.291851 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:34.269241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:37827)
0809 05:35:34.270068 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:34.270123 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:34.272163 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:34.272936 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:34.272945 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:34.273665 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:34.274503 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:34.274531 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.275678 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.275686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:34.277871 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:34.277889 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.278688 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.278698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:34.279060 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:34.279888 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:34.279924 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:34.283128 (+ 3204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:34.287609 (+ 4481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:34.287617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:34.287632 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:34.288254 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:34.288686 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:34.288690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:34.288692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:34.288893 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:34.289564 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:34.289569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:34.289922 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 05:35:34.290172 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 05:35:34.290603 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":367,"threads_started":1}
I20260809 05:35:34.293527 8618 heartbeater.cc:347] Connected to a master server at 127.30.137.254:46411
I20260809 05:35:34.293852 8618 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:34.294612 8618 heartbeater.cc:511] Master 127.30.137.254:46411 requested a full tablet report, sending...
I20260809 05:35:34.296885 8313 ts_manager.cc:194] Registered new tserver with Master: 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:34.298368 8313 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:37827
WARNING: no policy specified for oryx/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab.
Entry for principal oryx/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab.
I20260809 05:35:34.334071 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:46411
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:34.688913 8627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:34.689252 8627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:34.689459 8627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:34.697151 8627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:34.697309 8627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:34.697480 8627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:34.707916 8627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:46411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:34.709391 8627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:34.710825 8627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:34.725868 8633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:34.725936 8637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:34.725839 8634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:34.726215 8627 server_base.cc:1034] running on GCE node
I20260809 05:35:34.727433 8627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:34.728457 8627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:34.729812 8627 hybrid_clock.cc:648] HybridClock initialized: now 1786253734729745 us; error 69 us; skew 500 ppm
I20260809 05:35:34.730229 8627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:34 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253734, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:34.733606 8627 init.cc:377] Logged in from keytab as oryx/127.30.137.194@KRBTEST.COM (short username oryx)
I20260809 05:35:34.735148 8627 webserver.cc:460] Webserver started at http://127.30.137.194:45929/ using document root <none> and password file <none>
I20260809 05:35:34.735852 8627 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:34.736001 8627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:34.736332 8627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:34.739395 8627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "562c5edd49794b60b7fedb777ea228ab"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "f5634aeb5ab73b399c60310ca7ef5be0"
server_key_iv: "e1a4a4eb2a9241fe35f9d42c70c3673d"
server_key_version: "encryptionkey@0"
I20260809 05:35:34.740229 8627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "562c5edd49794b60b7fedb777ea228ab"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "f5634aeb5ab73b399c60310ca7ef5be0"
server_key_iv: "e1a4a4eb2a9241fe35f9d42c70c3673d"
server_key_version: "encryptionkey@0"
I20260809 05:35:34.745584 8627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:35:34.749235 8646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:34.750617 8627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:35:34.750850 8627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "562c5edd49794b60b7fedb777ea228ab"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "f5634aeb5ab73b399c60310ca7ef5be0"
server_key_iv: "e1a4a4eb2a9241fe35f9d42c70c3673d"
server_key_version: "encryptionkey@0"
I20260809 05:35:34.751073 8627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:34.783332 8627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:34.788409 8627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:34.788779 8627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:34.789871 8627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:34.791420 8627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:34.791523 8627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:34.791678 8627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:34.791760 8627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:34.827822 8627 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:35907
I20260809 05:35:34.827914 8827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:35907 every 8 connection(s)
I20260809 05:35:34.829186 8627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:34.832787 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 8627
I20260809 05:35:34.832993 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:34.833459 31271 external_mini_cluster.cc:1428] Setting key df4960c1709d1113b64a1b268dc571ca
Aug 09 05:35:34 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253734, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:34.850610 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:34.832103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:33201 (local address 127.30.137.254:46411)
0809 05:35:34.832628 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:34.832635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:34.834734 (+ 2099us) server_negotiation.cc:407] Connection header received
0809 05:35:34.835581 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:34.835586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:34.835677 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:34.835792 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:34.837351 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.838093 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.840302 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.840736 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.840869 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:34.844786 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:34.844791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:34.844794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:34.844830 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:34.847508 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:34.848266 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:34.848270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:34.848273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:34.848416 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:34.848859 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:34.848864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:34.848867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:34.849127 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:34.849280 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:34.849976 (+ 696us) server_negotiation.cc:299] Negotiation successful
0809 05:35:34.850205 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":1857,"threads_started":1}
I20260809 05:35:34.851677 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:34.832563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.194:33201)
0809 05:35:34.833417 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:34.833471 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:34.835312 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:34.836053 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:34.836062 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:34.836594 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:34.837173 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:34.837191 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.838268 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.838272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:34.840154 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:34.840168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:34.840994 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:34.841002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:34.841255 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:34.842007 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:34.842029 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:34.844548 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:34.847654 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:34.847664 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:34.847682 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:34.848133 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:34.848532 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:34.848538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:34.848541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:34.848735 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:34.849257 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:34.849267 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:34.849618 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 05:35:34.849896 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 05:35:34.850447 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":394,"threads_started":1}
I20260809 05:35:34.853325 8828 heartbeater.cc:347] Connected to a master server at 127.30.137.254:46411
I20260809 05:35:34.853654 8828 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:34.854401 8828 heartbeater.cc:511] Master 127.30.137.254:46411 requested a full tablet report, sending...
I20260809 05:35:34.856076 8313 ts_manager.cc:194] Registered new tserver with Master: 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907)
I20260809 05:35:34.857517 8313 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:33201
WARNING: no policy specified for oryx/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab.
Entry for principal oryx/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab.
I20260809 05:35:34.895891 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:46411
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:35.261723 8836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:35.262032 8836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:35.262184 8836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:35.272779 8836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:35.273033 8836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:35.273238 8836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:35.283490 8836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:46411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:35.284930 8836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:35.286331 8836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 05:35:35.301752 8618 heartbeater.cc:503] Master 127.30.137.254:46411 was elected leader, sending a full tablet report...
W20260809 05:35:35.303879 8842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:35.303741 8843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:35.304911 8846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:35.308506 8836 server_base.cc:1034] running on GCE node
I20260809 05:35:35.309342 8836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:35.310524 8836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:35.311764 8836 hybrid_clock.cc:648] HybridClock initialized: now 1786253735311721 us; error 43 us; skew 500 ppm
I20260809 05:35:35.312126 8836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:35 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253735, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:35.315819 8836 init.cc:377] Logged in from keytab as oryx/127.30.137.195@KRBTEST.COM (short username oryx)
I20260809 05:35:35.317673 8836 webserver.cc:460] Webserver started at http://127.30.137.195:35847/ using document root <none> and password file <none>
I20260809 05:35:35.318481 8836 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:35.318624 8836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:35.319020 8836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:35.322274 8836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "abb2f48b70c44def90f83a95298d9215"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "6bdd6d4fc26a01f10b03a4d01fb06779"
server_key_iv: "41eb5f0197b958adf27212e0bccb8dcf"
server_key_version: "encryptionkey@0"
I20260809 05:35:35.323238 8836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "abb2f48b70c44def90f83a95298d9215"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "6bdd6d4fc26a01f10b03a4d01fb06779"
server_key_iv: "41eb5f0197b958adf27212e0bccb8dcf"
server_key_version: "encryptionkey@0"
I20260809 05:35:35.329083 8836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:35:35.333048 8860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:35.334472 8836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:35:35.334725 8836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "abb2f48b70c44def90f83a95298d9215"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "6bdd6d4fc26a01f10b03a4d01fb06779"
server_key_iv: "41eb5f0197b958adf27212e0bccb8dcf"
server_key_version: "encryptionkey@0"
I20260809 05:35:35.334964 8836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:35.353760 8836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:35.358922 8836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:35.359256 8836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:35.360345 8836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:35.361789 8836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:35.361899 8836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:35.362054 8836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:35.362128 8836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:35.397372 8836 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:35527
I20260809 05:35:35.397420 9032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:35527 every 8 connection(s)
I20260809 05:35:35.398800 8836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:35.405701 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 8836
I20260809 05:35:35.405908 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:35.406350 31271 external_mini_cluster.cc:1428] Setting key 41f74765e8402bdb21298efa359a4d53
Aug 09 05:35:35 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253735, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:35.420403 9036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:35.401625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:37989 (local address 127.30.137.254:46411)
0809 05:35:35.402110 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:35.402115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:35.403190 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 05:35:35.405159 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:35.405164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:35.405277 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:35.405399 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:35.407300 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.408369 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.409990 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.410421 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.410559 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:35.414283 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:35.414289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:35.414292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:35.414329 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:35.416978 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:35.417807 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:35.417815 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:35.417819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:35.417959 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:35.418439 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:35.418444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:35.418447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:35.418803 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:35.418973 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:35.419729 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 05:35:35.419973 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":237,"threads_started":1}
I20260809 05:35:35.421581 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:35.402198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:37989)
0809 05:35:35.403009 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:35.403051 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:35.404858 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:35.405657 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:35.405665 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:35.406298 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:35.407093 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:35.407123 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.408523 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.408528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:35.409839 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:35.409853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.410703 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.410715 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:35.410981 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:35.411671 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:35.411701 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:35.414025 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:35.417148 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:35.417157 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:35.417177 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:35.417648 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:35.418073 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:35.418079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:35.418082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:35.418316 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:35.418938 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:35.418946 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:35.419326 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 05:35:35.419624 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 05:35:35.420233 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":292,"threads_started":1}
I20260809 05:35:35.423094 9033 heartbeater.cc:347] Connected to a master server at 127.30.137.254:46411
I20260809 05:35:35.423391 9033 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:35.424113 9033 heartbeater.cc:511] Master 127.30.137.254:46411 requested a full tablet report, sending...
I20260809 05:35:35.425498 8313 ts_manager.cc:194] Registered new tserver with Master: abb2f48b70c44def90f83a95298d9215 (127.30.137.195:35527)
I20260809 05:35:35.426266 8313 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:37989
I20260809 05:35:35.434717 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.137.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.253.keytab.
Entry for principal oryx/127.30.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.253.keytab.
I20260809 05:35:35.498449 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:41497
--webserver_interface=127.30.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.253.keytab
--principal=oryx/127.30.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:46411,127.30.137.253:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:35.859877 9044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:35.860273 9044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:35.860430 9044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 05:35:35.860535 8828 heartbeater.cc:503] Master 127.30.137.254:46411 was elected leader, sending a full tablet report...
W20260809 05:35:35.868644 9044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:35.868819 9044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:35.868868 9044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:35.868921 9044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:35.868968 9044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:35.883750 9044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:46411,127.30.137.253:41497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.253.keytab
--principal=oryx/127.30.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
W20260809 05:35:35.884264 9044 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.137.254:46411,127.30.137.253:41497'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 05:35:35.885651 9044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:35.887249 9044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:35.900660 9055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:35.900451 9060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:35.900735 9054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:35.901822 9044 server_base.cc:1034] running on GCE node
I20260809 05:35:35.902783 9044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:35.904368 9044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:35.905629 9044 hybrid_clock.cc:648] HybridClock initialized: now 1786253735905547 us; error 67 us; skew 500 ppm
I20260809 05:35:35.906002 9044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:35 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253735, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:35.909690 9044 init.cc:377] Logged in from keytab as oryx/127.30.137.253@KRBTEST.COM (short username oryx)
I20260809 05:35:35.911466 9044 webserver.cc:460] Webserver started at http://127.30.137.253:32807/ using document root <none> and password file <none>
I20260809 05:35:35.912220 9044 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:35.912318 9044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:35.912600 9044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:35.915940 9044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/instance:
uuid: "a73393a6b7f24427981fb718086a93d3"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "16519c141bccf43b2b6465a54bd4f598"
server_key_iv: "30eab8008d6ef6289a138308cfcf9694"
server_key_version: "encryptionkey@0"
I20260809 05:35:35.916762 9044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal/instance:
uuid: "a73393a6b7f24427981fb718086a93d3"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "16519c141bccf43b2b6465a54bd4f598"
server_key_iv: "30eab8008d6ef6289a138308cfcf9694"
server_key_version: "encryptionkey@0"
I20260809 05:35:35.922423 9044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:35:35.926156 9071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:35.927690 9044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:35.927902 9044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
uuid: "a73393a6b7f24427981fb718086a93d3"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "16519c141bccf43b2b6465a54bd4f598"
server_key_iv: "30eab8008d6ef6289a138308cfcf9694"
server_key_version: "encryptionkey@0"
I20260809 05:35:35.928141 9044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:35.947687 9044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:35.952577 9044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:35.952896 9044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:35.973378 9044 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.253:41497
I20260809 05:35:35.973393 9162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.253:41497 every 8 connection(s)
I20260809 05:35:35.974990 9044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
I20260809 05:35:35.976565 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 9044
I20260809 05:35:35.979943 9163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:35.988919 9163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:35:35 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
Aug 09 05:35:35 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253735, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.253@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:36.004076 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:35.979156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.30.137.253:41497)
0809 05:35:35.979963 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:35.979974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:35.980043 (+ 69us) server_negotiation.cc:407] Connection header received
0809 05:35:35.981716 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:35.981739 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:35.982122 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:35.982481 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:35.984567 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.985990 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.987961 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.988805 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.988949 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:35.994526 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:35.994544 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:35.994560 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:35.994621 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:35.998577 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:35.999451 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:35.999459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:35.999467 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:35.999665 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.000155 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.000159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.000161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.000935 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.002020 (+ 1085us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.002283 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.002813 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":385,"threads_started":1}
I20260809 05:35:36.009658 9168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:35.992473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36624 (local address 127.30.137.254:46411)
0809 05:35:35.993010 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:35.993017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:35.993616 (+ 599us) server_negotiation.cc:407] Connection header received
0809 05:35:35.993835 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:35.993842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:35.993951 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:35.994092 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:35.995829 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.996851 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.999048 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.999451 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.999597 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.003445 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.003452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.003455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.003498 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.006972 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.007890 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.007894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.007897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.008031 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.008384 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.008387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.008389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.008700 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.008814 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.009248 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.009419 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":263,"threads_started":1}
I20260809 05:35:36.009694 9170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:35.992426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:36624)
0809 05:35:35.993448 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:35.993494 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:35.993685 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:35.994148 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:35.994153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:35.995250 (+ 1097us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:35.995635 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:35.995651 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.997028 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.997034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:35.998893 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:35.998905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:35.999652 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:35.999658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:35.999855 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.000619 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.000636 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.003222 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.007117 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.007127 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.007132 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.007753 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.008131 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.008134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.008137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.008287 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.008806 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.008812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.008976 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.009140 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.009404 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":581,"threads_started":1}
I20260809 05:35:36.011355 9163 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253735, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.253@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
I20260809 05:35:36.024010 9170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.012083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:48270)
0809 05:35:36.012287 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.012311 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.012408 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.012663 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.012667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.013014 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.013166 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.013175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.014168 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.014172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.015168 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.015181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.015879 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.015884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.016028 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.016678 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.016697 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.018873 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.021747 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.021751 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.021754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.022159 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.022522 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.022525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.022528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.022739 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.023241 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.023246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.023390 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.023512 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.023761 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 05:35:36.024215 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.012083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.30.137.253:41497)
0809 05:35:36.012241 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.012247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.012376 (+ 129us) server_negotiation.cc:407] Connection header received
0809 05:35:36.012493 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.012497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.012552 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.012630 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.013290 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.013999 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.015290 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.015713 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.015807 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.019043 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.019048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.019051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.019080 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.021559 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.022276 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.022280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.022282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.022423 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.022835 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.022838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.022840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.023109 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.023237 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.023619 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.023911 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":53}
I20260809 05:35:36.029155 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Bootstrap starting.
I20260809 05:35:36.032655 9163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:36.033849 9163 log.cc:824] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:36.036875 9163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: No bootstrap required, opened a new log
I20260809 05:35:36.044334 9163 raft_consensus.cc:348] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } }
I20260809 05:35:36.044689 9163 raft_consensus.cc:374] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:36.044811 9163 raft_consensus.cc:729] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73393a6b7f24427981fb718086a93d3, State: Initialized, Role: FOLLOWER
I20260809 05:35:36.045533 9163 consensus_queue.cc:260] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } }
I20260809 05:35:36.046504 9177 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } } }
I20260809 05:35:36.046962 9177 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:35:36.047629 9163 sys_catalog.cc:564] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 05:35:36.057055 9195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:35:36.057288 9195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:35:36.059932 9044 master_runner.cc:428] Detected that this master a73393a6b7f24427981fb718086a93d3 is joining an existing cluster
I20260809 05:35:36.060066 9044 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.137.253:41497
I20260809 05:35:36.088141 9168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.066138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.30.137.254:46411)
0809 05:35:36.066372 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.066381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.067361 (+ 980us) server_negotiation.cc:407] Connection header received
0809 05:35:36.067706 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.067714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.067825 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.067944 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.070027 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.071515 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.075793 (+ 4278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.076522 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.076634 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.079750 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.079758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.079764 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.079810 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.084265 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.085263 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.085270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.085274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.085732 (+ 458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.086237 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.086242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.086245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.086687 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.086836 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.087468 (+ 632us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.087753 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:35:36.089634 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.066133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:36632)
0809 05:35:36.067242 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.067263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.067501 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.068251 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.068256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.069216 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.069783 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.069801 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.071781 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.071790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.073894 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.073908 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.076985 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.076998 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.077193 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.078113 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.078137 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.079469 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.084470 (+ 5001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.084477 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.084485 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.085091 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.085904 (+ 813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.085909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.085913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.086093 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.086921 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.086926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.087158 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.088739 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.089131 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":30,"thread_start_us":824,"threads_started":1}
I20260809 05:35:36.091086 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.066867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.30.137.253:41497)
0809 05:35:36.067109 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.067118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.073990 (+ 6872us) server_negotiation.cc:407] Connection header received
0809 05:35:36.074176 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.074184 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.074268 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.074388 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.075718 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.076971 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.079322 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.080183 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.080295 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.083197 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.083206 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.083214 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.083255 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.087606 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.088571 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.088579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.088584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.088754 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.089239 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.089245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.089249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.089666 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.089839 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.090416 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.090690 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for oryx/127.30.137.252@KRBTEST.COM; defaulting to no policy
I20260809 05:35:36.092525 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.067541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:48272)
0809 05:35:36.073859 (+ 6318us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.073881 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.074037 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.074440 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.074448 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.075199 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.075543 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.075556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.077189 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.077196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.079137 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.079159 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.080632 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.080641 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.080862 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.081623 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.081650 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.082936 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.087772 (+ 4836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.087779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.087786 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.088433 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.088908 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.088915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.088920 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.089122 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.089913 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.089921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.090088 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.091631 (+ 1543us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.091953 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6126,"mutex_wait_us":32,"thread_start_us":5903,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.252@KRBTEST.COM" created.
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.254@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
I20260809 05:35:36.128074 9207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.097918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:48280)
0809 05:35:36.107351 (+ 9433us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.107401 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.107772 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.108348 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.108357 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.109329 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.109691 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.109710 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.111352 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.111360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.113320 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.113343 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.114486 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.114496 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.114799 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.115710 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.115751 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.119269 (+ 3518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.124433 (+ 5164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.124456 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.124465 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.125179 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.125740 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.125745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.125749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.125948 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.126708 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.126719 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.126961 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.127210 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.127622 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9130,"thread_start_us":8973,"threads_started":1}
I20260809 05:35:36.128113 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.098318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48280 (local address 127.30.137.253:41497)
0809 05:35:36.098519 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.098526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.107557 (+ 9031us) server_negotiation.cc:407] Connection header received
0809 05:35:36.108047 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.108055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.108158 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.108278 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.109920 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.111147 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.113530 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.114271 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.114430 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.119586 (+ 5156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.119596 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.119608 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.119670 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.124182 (+ 4512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.125360 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.125370 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.125377 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.125585 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.126089 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.126095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.126099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.126531 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.126740 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.127496 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.127736 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 05:35:36.130409 8313 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.137.253:41497: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a73393a6b7f24427981fb718086a93d3"
member_type: NON_VOTER
last_known_addr {
host: "127.30.137.253"
port: 41497
}
attrs {
promote: true
}
}
dest_uuid: "5503b5dfd58e4712910fac17b2f1da82"
cas_config_opid_index: -1
I20260809 05:35:36.132122 8313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } }
I20260809 05:35:36.136083 9213 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a73393a6b7f24427981fb718086a93d3 (127.30.137.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } }
I20260809 05:35:36.137638 9124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.252.keytab.
Entry for principal oryx/127.30.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.252.keytab.
I20260809 05:35:36.140954 9213 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.137.253:41497
I20260809 05:35:36.141774 9213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } }
I20260809 05:35:36.141984 9044 master.cc:537] Master@127.30.137.253:41497 shutting down...
I20260809 05:35:36.142024 9213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:36.142939 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:41165
--webserver_interface=127.30.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.252.keytab
--principal=oryx/127.30.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:35:36.144729 9124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Refusing update from remote peer 5503b5dfd58e4712910fac17b2f1da82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 05:35:36.146384 9177 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:36.146342 9213 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:36.146638 9177 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } }
I20260809 05:35:36.148059 9177 leader_election.cc:288] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411)
I20260809 05:35:36.149883 8330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a73393a6b7f24427981fb718086a93d3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5503b5dfd58e4712910fac17b2f1da82" is_pre_election: true
I20260809 05:35:36.151275 9072 leader_election.cc:302] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a73393a6b7f24427981fb718086a93d3; no voters: 5503b5dfd58e4712910fac17b2f1da82
W20260809 05:35:36.153103 8294 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 05:35:36.154789 9177 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 05:35:36.157657 8294 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 -> Peer a73393a6b7f24427981fb718086a93d3 (127.30.137.253:41497): Couldn't send request to peer a73393a6b7f24427981fb718086a93d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 05:35:36.161043 9044 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:36.161536 9044 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:36.161747 9044 tablet_replica.cc:330] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: stopping tablet replica
I20260809 05:35:36.170624 9044 master.cc:552] Master@127.30.137.253:41497 shutdown complete.
I20260809 05:35:36.170984 9044 master_runner.cc:305] Clearing existing system tablet
I20260809 05:35:36.171564 9044 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:36.171751 9044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:36.179505 9044 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.003s
I20260809 05:35:36.181851 9221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:36.182520 9044 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:36.183060 9044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 05:35:36.183218 9044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
uuid: "a73393a6b7f24427981fb718086a93d3"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "16519c141bccf43b2b6465a54bd4f598"
server_key_iv: "30eab8008d6ef6289a138308cfcf9694"
server_key_version: "encryptionkey@0"
I20260809 05:35:36.188905 9044 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:36.197100 9044 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 05:35:36.197500 9044 log.cc:1196] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 05:35:36.198160 9044 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Deleting consensus metadata
I20260809 05:35:36.198551 9044 master_runner.cc:315] Copying system tablet from 127.30.137.254:46411
I20260809 05:35:36.201743 9044 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: tablet copy: Beginning tablet copy session from remote peer at address 127.30.137.254:46411
I20260809 05:35:36.219642 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.202853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:36644)
0809 05:35:36.203462 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.203485 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.203748 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.204352 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.204357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.204958 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.205263 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.205282 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.206884 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.206890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.208482 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.208500 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.209555 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.209562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.209768 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.210614 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.210639 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.211876 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.216350 (+ 4474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.216364 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.216369 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.216944 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.217471 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.217481 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.217485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.217677 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.218382 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.218387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.218561 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.218773 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.219139 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":602,"threads_started":1}
I20260809 05:35:36.220664 9168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.203217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.30.137.254:46411)
0809 05:35:36.203439 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.203446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.203598 (+ 152us) server_negotiation.cc:407] Connection header received
0809 05:35:36.203930 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.203937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.204034 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.204109 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.205481 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.206645 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.208676 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.209354 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.209496 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.212139 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.212147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.212153 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.212204 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.216148 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.217106 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.217115 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.217121 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.217325 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.217819 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.217827 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.217832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.218239 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.218404 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.220073 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.220312 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:35:36.221503 8345 tablet_copy_service.cc:140] P 5503b5dfd58e4712910fac17b2f1da82: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a73393a6b7f24427981fb718086a93d3 ({username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:36644)
I20260809 05:35:36.221804 8345 tablet_copy_service.cc:161] P 5503b5dfd58e4712910fac17b2f1da82: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a73393a6b7f24427981fb718086a93d3 at {username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:36644: session id = a73393a6b7f24427981fb718086a93d3-00000000000000000000000000000000
I20260809 05:35:36.227552 8345 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 05:35:36.230341 9044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 05:35:36.237613 9044 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: tablet copy: Starting download of 0 data blocks...
I20260809 05:35:36.237990 9044 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: tablet copy: Starting download of 1 WAL segments...
I20260809 05:35:36.241073 9044 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 05:35:36.246112 8345 tablet_copy_service.cc:342] P 5503b5dfd58e4712910fac17b2f1da82: Request end of tablet copy session a73393a6b7f24427981fb718086a93d3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:36644
I20260809 05:35:36.246330 8345 tablet_copy_service.cc:433] P 5503b5dfd58e4712910fac17b2f1da82: ending tablet copy session a73393a6b7f24427981fb718086a93d3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a73393a6b7f24427981fb718086a93d3
I20260809 05:35:36.259068 9044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:36.262166 9232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:36.271759 9234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:36.276546 9237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:36.276894 9044 server_base.cc:1034] running on GCE node
I20260809 05:35:36.277457 9044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:36.278203 9044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:36.279536 9044 hybrid_clock.cc:648] HybridClock initialized: now 1786253736279457 us; error 43 us; skew 500 ppm
I20260809 05:35:36.279788 9044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253736, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:36.282155 9044 init.cc:377] Logged in from keytab as oryx/127.30.137.253@KRBTEST.COM (short username oryx)
I20260809 05:35:36.288252 9044 webserver.cc:460] Webserver started at http://127.30.137.253:37639/ using document root <none> and password file <none>
I20260809 05:35:36.288774 9044 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:36.288878 9044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:36.292382 9044 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 05:35:36.294750 9245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:36.295331 9044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 05:35:36.295477 9044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
uuid: "a73393a6b7f24427981fb718086a93d3"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "16519c141bccf43b2b6465a54bd4f598"
server_key_iv: "30eab8008d6ef6289a138308cfcf9694"
server_key_version: "encryptionkey@0"
I20260809 05:35:36.295655 9044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:36.310905 9044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:36.315140 9044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:36.343708 9044 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.253:41497
I20260809 05:35:36.343794 9327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.253:41497 every 8 connection(s)
I20260809 05:35:36.344852 9044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
I20260809 05:35:36.350713 9329 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:35:36.354710 9329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Bootstrap starting.
I20260809 05:35:36.375109 9329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:35:36.375857 9329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Bootstrap complete.
I20260809 05:35:36.377368 9329 raft_consensus.cc:348] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } }
I20260809 05:35:36.377750 9329 raft_consensus.cc:729] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a73393a6b7f24427981fb718086a93d3, State: Initialized, Role: LEARNER
I20260809 05:35:36.378087 9329 consensus_queue.cc:260] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } }
I20260809 05:35:36.379132 9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } }
I20260809 05:35:36.379457 9329 sys_catalog.cc:564] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:36.379650 9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: LEARNER
I20260809 05:35:36.390601 9349 catalog_manager.cc:1259] Loaded cluster ID: e3949a3c8a6845f29237bf3225b0be8a
I20260809 05:35:36.390827 9349 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: loading cluster ID for follower catalog manager: success
I20260809 05:35:36.394600 9349 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: acquiring CA information for follower catalog manager: success
I20260809 05:35:36.397338 9349 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:35:36.428886 9033 heartbeater.cc:503] Master 127.30.137.254:46411 was elected leader, sending a full tablet report...
W20260809 05:35:36.571698 9216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:36.572110 9216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:36.572264 9216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:36.580035 9216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:36.580219 9216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:36.580318 9216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:36.580420 9216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:36.580551 9216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:36.592057 9216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.252.keytab
--principal=oryx/127.30.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:41165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:36.593573 9216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:36.595088 9216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:36.607806 9357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:36.607708 9356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:36.607837 9360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:36.608695 9216 server_base.cc:1034] running on GCE node
I20260809 05:35:36.609483 9216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:36.611429 9216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:36.612699 9216 hybrid_clock.cc:648] HybridClock initialized: now 1786253736612634 us; error 62 us; skew 500 ppm
I20260809 05:35:36.613123 9216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253736, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:36.616648 9216 init.cc:377] Logged in from keytab as oryx/127.30.137.252@KRBTEST.COM (short username oryx)
I20260809 05:35:36.618191 9216 webserver.cc:460] Webserver started at http://127.30.137.252:37843/ using document root <none> and password file <none>
I20260809 05:35:36.618949 9216 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:36.619063 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:36.619330 9216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:36.622429 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/instance:
uuid: "62d9b52c7f974c8785e70bb8a805db42"
format_stamp: "Formatted at 2026-08-09 05:35:36 on dist-test-slave-3kk6"
server_key: "a41dec90759932f515ae44578ae63f62"
server_key_iv: "c61326baef6292173e0c13d3875d3a7e"
server_key_version: "encryptionkey@0"
I20260809 05:35:36.623358 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal/instance:
uuid: "62d9b52c7f974c8785e70bb8a805db42"
format_stamp: "Formatted at 2026-08-09 05:35:36 on dist-test-slave-3kk6"
server_key: "a41dec90759932f515ae44578ae63f62"
server_key_iv: "c61326baef6292173e0c13d3875d3a7e"
server_key_version: "encryptionkey@0"
I20260809 05:35:36.630340 9216 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 05:35:36.635007 9368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:36.636747 9216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 05:35:36.636945 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
uuid: "62d9b52c7f974c8785e70bb8a805db42"
format_stamp: "Formatted at 2026-08-09 05:35:36 on dist-test-slave-3kk6"
server_key: "a41dec90759932f515ae44578ae63f62"
server_key_iv: "c61326baef6292173e0c13d3875d3a7e"
server_key_version: "encryptionkey@0"
I20260809 05:35:36.637173 9216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:36.651400 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.643861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.30.137.253:41497)
0809 05:35:36.644532 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.644540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.645112 (+ 572us) server_negotiation.cc:407] Connection header received
0809 05:35:36.645356 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.645364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.645506 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.645706 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:36.646166 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.647286 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.649551 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.650369 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.650518 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.650731 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.650958 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.651119 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":330,"threads_started":1}
I20260809 05:35:36.651414 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.643767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:48294)
0809 05:35:36.644951 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.644973 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.645180 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.645721 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.645725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.645755 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:36.646014 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.646025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.647435 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.647440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.649374 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.649391 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.650537 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.650542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.650748 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.650819 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.650911 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.651065 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1023,"thread_start_us":912,"threads_started":1}
I20260809 05:35:36.652518 9295 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 05:35:36.653120 9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5503b5dfd58e4712910fac17b2f1da82. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } }
I20260809 05:35:36.653427 9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: LEARNER
I20260809 05:35:36.680505 9216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:36.685461 9216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:36.685782 9216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:36.706885 9216 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.252:41165
I20260809 05:35:36.706910 9442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.252:41165 every 8 connection(s)
I20260809 05:35:36.708354 9216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
I20260809 05:35:36.712421 9443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:36.713335 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 9216
I20260809 05:35:36.722746 9443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.252@KRBTEST.COM
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253736, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.252@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:36.740540 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.715760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.30.137.252:41165)
0809 05:35:36.716574 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.716592 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.716645 (+ 53us) server_negotiation.cc:407] Connection header received
0809 05:35:36.719390 (+ 2745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.719426 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.720004 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.720514 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.722076 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.724209 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.725534 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.726547 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.726778 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.732268 (+ 5490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.732278 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.732292 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.732337 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.735656 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.736386 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.736395 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.736409 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.736568 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.736971 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.736976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.736979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.737656 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.738010 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.738458 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.739119 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":325,"threads_started":1}
I20260809 05:35:36.743963 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.725906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.30.137.254:46411)
0809 05:35:36.726554 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.726564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.727361 (+ 797us) server_negotiation.cc:407] Connection header received
0809 05:35:36.727578 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.727586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.727669 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.727759 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.729294 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.730080 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.732134 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.732826 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.732996 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.736560 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.736566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.736571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.736612 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.739590 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.740439 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.740444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.740448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.740592 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.742186 (+ 1594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.742190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.742192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.742562 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.742809 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.743358 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.743621 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":331,"threads_started":1}
I20260809 05:35:36.744053 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.725821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:36650)
0809 05:35:36.727201 (+ 1380us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.727261 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.727417 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.727865 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.727873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.728780 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.729145 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.729159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.730269 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.730277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.731959 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.731973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.733046 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.733055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.733284 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.733952 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.733978 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.736371 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.739726 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.739737 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.739744 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.740303 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.741796 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.741805 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.741809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.742016 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.742748 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.742755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.742972 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.743214 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.743567 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"thread_start_us":981,"threads_started":1}
I20260809 05:35:36.745498 9443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253736, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.252@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
I20260809 05:35:36.758273 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.746382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:48310)
0809 05:35:36.746707 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.746728 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.746832 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.747100 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.747104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.747479 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.747617 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.747625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.748633 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.748637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.749591 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.749603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.750278 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.750283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.750417 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.751056 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.751073 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.753166 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.756173 (+ 3007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.756178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.756184 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.756590 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.756948 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.756951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.756954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.757081 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.757522 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.757526 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.757638 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.757742 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.757990 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 05:35:36.758550 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.746388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.30.137.253:41497)
0809 05:35:36.746550 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.746554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.746797 (+ 243us) server_negotiation.cc:407] Connection header received
0809 05:35:36.746920 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.746924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.746985 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.747068 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.747739 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.748486 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.749715 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.750118 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.750208 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.753341 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.753346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.753350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.753385 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.756043 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.756705 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.756709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.756711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.756841 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.757165 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.757168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.757171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.757431 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.757541 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.757964 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.758210 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 05:35:36.759548 9443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253736, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.252@KRBTEST.COM for oryx/127.30.137.252@KRBTEST.COM
I20260809 05:35:36.772485 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.760221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:52296)
0809 05:35:36.760420 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.760437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.760524 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.760784 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.760787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.761135 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.761280 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.761288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.762229 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.762233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.763229 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.763242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.763907 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.763912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.764073 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.764637 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.764655 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.766837 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.769977 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.769982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.769984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.770408 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.770790 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.770794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.770796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.770953 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.771549 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.771553 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.771702 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.771834 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.772040 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 05:35:36.772526 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.760301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.30.137.252:41165)
0809 05:35:36.760456 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.760460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.760475 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:35:36.760614 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.760617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.760671 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.760750 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.761402 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.762092 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.763354 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.763760 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.763860 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.767034 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.767038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.767042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.767068 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.769808 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.770524 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.770528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.770530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.770644 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.771047 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.771050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.771052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.771433 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.771597 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.771927 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.772163 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":58}
I20260809 05:35:36.777431 9443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Bootstrap starting.
I20260809 05:35:36.780907 9443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:36.782016 9443 log.cc:824] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:36.784946 9443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: No bootstrap required, opened a new log
I20260809 05:35:36.792443 9443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } }
I20260809 05:35:36.792826 9443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:36.792922 9443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d9b52c7f974c8785e70bb8a805db42, State: Initialized, Role: FOLLOWER
I20260809 05:35:36.793447 9443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } }
I20260809 05:35:36.794486 9459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } } }
I20260809 05:35:36.795058 9459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:35:36.796088 9443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 05:35:36.806042 9479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:35:36.806198 9479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:35:36.810616 9216 master_runner.cc:428] Detected that this master 62d9b52c7f974c8785e70bb8a805db42 is joining an existing cluster
I20260809 05:35:36.810809 9216 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.137.252:41165
I20260809 05:35:36.834823 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.816560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.30.137.252:41165)
0809 05:35:36.816753 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.816760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.818700 (+ 1940us) server_negotiation.cc:407] Connection header received
0809 05:35:36.818880 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.818887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.818975 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.819113 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.820675 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.821929 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.824030 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.824968 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.825098 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.827626 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.827632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.827636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.827667 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.831124 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832196 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.832201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.832214 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.832375 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832812 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.832817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.832821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.833307 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.833465 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.834078 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.834398 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":63}
I20260809 05:35:36.834812 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.815043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.30.137.253:41497)
0809 05:35:36.815238 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.815245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.816426 (+ 1181us) server_negotiation.cc:407] Connection header received
0809 05:35:36.817158 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.817164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.817256 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.817330 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.820012 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.821047 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.823206 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.823866 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.823962 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.827411 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.827420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.827424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.827462 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.831227 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832336 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.832342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.832346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.832499 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.833006 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.833014 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.833020 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.833383 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.833522 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.834201 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.834435 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:35:36.835891 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.816248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:52304)
0809 05:35:36.818495 (+ 2247us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.818512 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.818749 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.819359 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.819364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.820144 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.820511 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.820524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.822153 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.822163 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.823862 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.823877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.825161 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.825172 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.825422 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.826145 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.826167 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.827456 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.831341 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.831347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.831353 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.832065 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.832514 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832518 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.832522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.832680 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.833542 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.833548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.833780 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.834142 (+ 362us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.834604 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2122,"mutex_wait_us":168,"thread_start_us":1915,"threads_started":1}
I20260809 05:35:36.836124 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.815058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.30.137.254:46411)
0809 05:35:36.815238 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.815245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.817073 (+ 1828us) server_negotiation.cc:407] Connection header received
0809 05:35:36.817303 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.817310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.817419 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.817494 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.819377 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.820598 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.823045 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.823717 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.823829 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.827235 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.827244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.827249 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.827295 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.831959 (+ 4664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832800 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.832806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.832810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.832976 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.833487 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.833493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.833496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.833857 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.834003 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.835534 (+ 1531us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.835785 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:35:36.835963 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.815244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:48312)
0809 05:35:36.816141 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.816169 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.816979 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.817559 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.817564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.819387 (+ 1823us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.819804 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.819822 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.821252 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.821259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.823060 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.823072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.824253 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.824259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.824767 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.825848 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.825883 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.827180 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.831398 (+ 4218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.831409 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.831414 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.832194 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.832655 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.832663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.832850 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.833606 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.833611 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.833899 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.835091 (+ 1192us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.835447 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":102,"thread_start_us":489,"threads_started":1}
I20260809 05:35:36.837107 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.815262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:36660)
0809 05:35:36.816967 (+ 1705us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.816983 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.817162 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.817720 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.817727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.818730 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.819177 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.819192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.820821 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.820826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.822860 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.822875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.824160 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.824171 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.824396 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.825435 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.825458 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.827001 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.832153 (+ 5152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.832157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.832160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.832659 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.833128 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.833132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.833135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.833332 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.834086 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.834089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.835207 (+ 1118us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.836323 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.836651 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1582,"mutex_wait_us":111,"spinlock_wait_cycles":78848,"thread_start_us":1318,"threads_started":1}
I20260809 05:35:36.847486 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 8408
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.254@KRBTEST.COM for oryx/127.30.137.252@KRBTEST.COM
Aug 09 05:35:36 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.193@KRBTEST.COM
I20260809 05:35:36.859591 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.842719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52310 (local address 127.30.137.252:41165)
0809 05:35:36.842910 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.842916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.843037 (+ 121us) server_negotiation.cc:407] Connection header received
0809 05:35:36.843256 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.843265 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.843350 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.843469 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.844281 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.845038 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.846428 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.847219 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.847348 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.851959 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.851966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.851971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.852015 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.856218 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.857219 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.857227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.857235 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.857395 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.857854 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.857859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.857862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.858262 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.858429 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.858960 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.859214 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:35:36.859622 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.842406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:52310)
0809 05:35:36.842947 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.842966 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.843100 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.843491 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.843498 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.843996 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.844142 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.844151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.845184 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.845189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.846237 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.846250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.847380 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.847387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.847565 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.848253 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.848279 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.851749 (+ 3470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.856364 (+ 4615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.856370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.856374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.857069 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.857527 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.857533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.857538 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.857722 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.858409 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.858416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.858691 (+ 275us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.858873 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.859178 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405}
I20260809 05:35:36.861534 8313 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.137.252:41165: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42"
member_type: NON_VOTER
last_known_addr {
host: "127.30.137.252"
port: 41165
}
attrs {
promote: true
}
}
dest_uuid: "5503b5dfd58e4712910fac17b2f1da82"
cas_config_opid_index: 5
I20260809 05:35:36.862834 8313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } }
I20260809 05:35:36.864895 9213 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.865404 9409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:36.867478 9295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 LEARNER]: Refusing update from remote peer 5503b5dfd58e4712910fac17b2f1da82: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 05:35:36.871052 9409 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Refusing update from remote peer 5503b5dfd58e4712910fac17b2f1da82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 05:35:36.874351 9492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.848528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.30.137.193:41035)
0809 05:35:36.849188 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.849195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.849220 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:35:36.849281 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.849287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.849499 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.849692 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.851351 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.852853 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.854162 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.854881 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.855098 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.860972 (+ 5874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.860990 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.861009 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.861053 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.868220 (+ 7167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.869436 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.869452 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.869458 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.869680 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.870255 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.870263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.870270 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.870882 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.872985 (+ 2103us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.873203 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.873590 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":313,"threads_started":1}
I20260809 05:35:36.875991 8611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:36.881265 9495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 05:35:36.881783 9213 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.137.252:41165
I20260809 05:35:36.882522 9216 master.cc:537] Master@127.30.137.252:41165 shutting down...
I20260809 05:35:36.883713 9213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.884131 9213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:36.889895 9493 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:36.896543 9216 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:36.897099 9216 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:36.897361 9216 tablet_replica.cc:330] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: stopping tablet replica
W20260809 05:35:36.912886 8294 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 -> Peer 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252:41165): Couldn't send request to peer 62d9b52c7f974c8785e70bb8a805db42. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 05:35:36.916402 9216 master.cc:552] Master@127.30.137.252:41165 shutdown complete.
I20260809 05:35:36.915761 9295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.916502 9216 master_runner.cc:305] Clearing existing system tablet
I20260809 05:35:36.916922 9216 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:36.917050 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:36.921267 9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: NON_VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: true } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.921607 9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: LEARNER
I20260809 05:35:36.922786 9213 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: attempting to promote NON_VOTER a73393a6b7f24427981fb718086a93d3 to VOTER
I20260809 05:35:36.924185 9213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } }
I20260809 05:35:36.926776 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.926097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:52324)
0809 05:35:36.926410 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.926555 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:41165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260809 05:35:36.926843 9295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 LEARNER]: Refusing update from remote peer 5503b5dfd58e4712910fac17b2f1da82: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 05:35:36.927335 9216 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.007s sys 0.000s
I20260809 05:35:36.927874 9369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.927457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:52326)
0809 05:35:36.927615 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.927703 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:41165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 05:35:36.927788 9213 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 05:35:36.928161 8294 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 -> Peer 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252:41165): Couldn't send request to peer 62d9b52c7f974c8785e70bb8a805db42. Status: Network error: Client connection negotiation failed: client connection to 127.30.137.252:41165: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 05:35:36.930199 9493 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a73393a6b7f24427981fb718086a93d3 (127.30.137.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.935360 9295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a73393a6b7f24427981fb718086a93d3 (127.30.137.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.936643 9505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:36.937049 9216 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:36.937467 9216 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.007s
I20260809 05:35:36.937625 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
uuid: "62d9b52c7f974c8785e70bb8a805db42"
format_stamp: "Formatted at 2026-08-09 05:35:36 on dist-test-slave-3kk6"
server_key: "a41dec90759932f515ae44578ae63f62"
server_key_iv: "c61326baef6292173e0c13d3875d3a7e"
server_key_version: "encryptionkey@0"
I20260809 05:35:36.940084 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.940387 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:36.940270 9495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.940595 9495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:36.940963 9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:36.941258 9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:35:36.944470 9216 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:36.951529 9216 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 05:35:36.951766 9216 log.cc:1196] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 05:35:36.952328 9216 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Deleting consensus metadata
I20260809 05:35:36.952628 9216 master_runner.cc:315] Copying system tablet from 127.30.137.254:46411
I20260809 05:35:36.955420 9216 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: tablet copy: Beginning tablet copy session from remote peer at address 127.30.137.254:46411
I20260809 05:35:36.972353 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.956330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:36674)
0809 05:35:36.956959 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:36.956979 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:36.957202 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:36.957715 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:36.957720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:36.958311 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:36.958581 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.958596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.960187 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.960193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.961727 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:36.961742 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.962850 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.962857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:36.963059 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.963886 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:36.963912 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:36.965049 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:36.969314 (+ 4265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.969320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.969324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.969851 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.970255 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:36.970261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:36.970266 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:36.970461 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:36.971084 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:36.971089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:36.971326 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 05:35:36.971490 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 05:35:36.971853 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":302,"threads_started":1}
I20260809 05:35:36.973031 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:36.956397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.30.137.254:46411)
0809 05:35:36.956635 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:36.956643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:36.957083 (+ 440us) server_negotiation.cc:407] Connection header received
0809 05:35:36.957344 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:36.957350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:36.957443 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:36.957513 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:36.958797 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.959992 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.961907 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:36.962575 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:36.962692 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:36.965281 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:36.965288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:36.965293 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:36.965334 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:36.969142 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.969973 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.969979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.969983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.970133 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:36.970576 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:36.970581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:36.970585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:36.970959 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:36.971101 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:36.971620 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 05:35:36.972339 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 05:35:36.973681 8345 tablet_copy_service.cc:140] P 5503b5dfd58e4712910fac17b2f1da82: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 62d9b52c7f974c8785e70bb8a805db42 ({username='oryx', principal='oryx/127.30.137.252@KRBTEST.COM'} at 127.0.0.1:36674)
I20260809 05:35:36.973842 8345 tablet_copy_service.cc:161] P 5503b5dfd58e4712910fac17b2f1da82: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 62d9b52c7f974c8785e70bb8a805db42 at {username='oryx', principal='oryx/127.30.137.252@KRBTEST.COM'} at 127.0.0.1:36674: session id = 62d9b52c7f974c8785e70bb8a805db42-00000000000000000000000000000000
I20260809 05:35:36.977509 8345 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 05:35:36.979485 9216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 05:35:36.986729 9216 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: tablet copy: Starting download of 0 data blocks...
I20260809 05:35:36.987144 9216 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: tablet copy: Starting download of 1 WAL segments...
I20260809 05:35:36.989627 9216 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 05:35:36.994325 8345 tablet_copy_service.cc:342] P 5503b5dfd58e4712910fac17b2f1da82: Request end of tablet copy session 62d9b52c7f974c8785e70bb8a805db42-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.137.252@KRBTEST.COM'} at 127.0.0.1:36674
I20260809 05:35:36.994469 8345 tablet_copy_service.cc:433] P 5503b5dfd58e4712910fac17b2f1da82: ending tablet copy session 62d9b52c7f974c8785e70bb8a805db42-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 62d9b52c7f974c8785e70bb8a805db42
I20260809 05:35:37.006713 9216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:37.009577 9523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:37.010063 9524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:37.015642 9526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:37.016448 9216 server_base.cc:1034] running on GCE node
I20260809 05:35:37.016855 9216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:37.016921 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 8408
I20260809 05:35:37.038564 9216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:37.039821 9216 hybrid_clock.cc:648] HybridClock initialized: now 1786253737039778 us; error 62 us; skew 500 ppm
I20260809 05:35:37.040132 9216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:37 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253737, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:37.042181 9216 init.cc:377] Logged in from keytab as oryx/127.30.137.252@KRBTEST.COM (short username oryx)
I20260809 05:35:37.047691 9216 webserver.cc:460] Webserver started at http://127.30.137.252:41717/ using document root <none> and password file <none>
I20260809 05:35:37.048008 9216 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:37.048123 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:37.050627 9216 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 05:35:37.052412 9535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:37.053064 9216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 05:35:37.053202 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
uuid: "62d9b52c7f974c8785e70bb8a805db42"
format_stamp: "Formatted at 2026-08-09 05:35:36 on dist-test-slave-3kk6"
server_key: "a41dec90759932f515ae44578ae63f62"
server_key_iv: "c61326baef6292173e0c13d3875d3a7e"
server_key_version: "encryptionkey@0"
I20260809 05:35:37.053429 9216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:37.054394 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:41035
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:46411
--webserver_port=45823
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:35:37.068050 9216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:37.071893 9216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:37.113696 9216 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.252:41165
I20260809 05:35:37.113762 9620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.252:41165 every 8 connection(s)
I20260809 05:35:37.114768 9216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
I20260809 05:35:37.122115 9621 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:35:37.127557 9621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Bootstrap starting.
I20260809 05:35:37.144256 9621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:35:37.144799 9621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: Bootstrap complete.
I20260809 05:35:37.145840 9621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } }
I20260809 05:35:37.146215 9621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 62d9b52c7f974c8785e70bb8a805db42, State: Initialized, Role: LEARNER
I20260809 05:35:37.146481 9621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } }
I20260809 05:35:37.147437 9625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:37.147868 9621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:37.147987 9625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: This master's current role is: LEARNER
I20260809 05:35:37.159812 9638 catalog_manager.cc:1259] Loaded cluster ID: e3949a3c8a6845f29237bf3225b0be8a
I20260809 05:35:37.159948 9638 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: loading cluster ID for follower catalog manager: success
I20260809 05:35:37.163800 9638 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: acquiring CA information for follower catalog manager: success
I20260809 05:35:37.166381 9638 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 05:35:37.450213 9544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:37.450521 9544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:37.450752 9544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:37.458611 9544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:37.458839 9544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:37.459043 9544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:37.469708 9544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:41035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=45823
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:37.471349 9544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:37.472810 9544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:37.487563 9646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:37.487422 9650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:37.487576 9647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:37.488523 9544 server_base.cc:1034] running on GCE node
I20260809 05:35:37.489157 9544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:37.490063 9544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:37.491263 9544 hybrid_clock.cc:648] HybridClock initialized: now 1786253737491200 us; error 55 us; skew 500 ppm
I20260809 05:35:37.491581 9544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:37 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253737, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:37.495128 9544 init.cc:377] Logged in from keytab as oryx/127.30.137.193@KRBTEST.COM (short username oryx)
I20260809 05:35:37.496835 9544 webserver.cc:460] Webserver started at http://127.30.137.193:45823/ using document root <none> and password file <none>
I20260809 05:35:37.497584 9544 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:37.497754 9544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:37.503432 9544 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 05:35:37.506235 9661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:37.507679 9544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:35:37.507874 9544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "938f3ee41234415bb7818fd76ff7b626"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "8cf454d0eb0089897834a4f22598620f"
server_key_iv: "3d22ff9ab6edb43b71ec028e4466ded6"
server_key_version: "encryptionkey@0"
I20260809 05:35:37.508927 9544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:37.521448 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.514306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:52334)
0809 05:35:37.515108 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:37.515135 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:37.515342 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:37.516034 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:37.516037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:37.516062 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:37.516289 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.516298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.518069 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.518073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.519311 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.519325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.520874 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.520879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.521012 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.521038 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 05:35:37.521114 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 05:35:37.521220 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":490,"threads_started":1}
I20260809 05:35:37.521792 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.514523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.30.137.252:41165)
0809 05:35:37.515686 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:37.515692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:37.515715 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:35:37.515790 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:37.515795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:37.515880 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:37.516003 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:37.516432 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.517907 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.519457 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.520712 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.520875 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.521052 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:37.521138 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 05:35:37.521278 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1005,"thread_start_us":875,"threads_started":1}
I20260809 05:35:37.522616 9583 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 05:35:37.523257 9625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5503b5dfd58e4712910fac17b2f1da82. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:37.523541 9625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: This master's current role is: LEARNER
I20260809 05:35:37.537595 9544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:37.542160 9544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:37.542476 9544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:37.543601 9544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:37.545034 9544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:37.545156 9544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:37.545285 9544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:37.545382 9544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:37.582926 9544 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:41035
I20260809 05:35:37.583037 9846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:41035 every 8 connection(s)
I20260809 05:35:37.584527 9544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:37.588199 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 9544
I20260809 05:35:37.588598 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 8627
Aug 09 05:35:37 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253737, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
Aug 09 05:35:37 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253737, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.252@KRBTEST.COM
Aug 09 05:35:37 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253737, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
Aug 09 05:35:37 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.194@KRBTEST.COM
I20260809 05:35:37.630352 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.604187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:45463 (local address 127.30.137.252:41165)
0809 05:35:37.604395 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:37.604402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:37.605264 (+ 862us) server_negotiation.cc:407] Connection header received
0809 05:35:37.606127 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:37.606133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:37.606232 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:37.606310 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:37.610254 (+ 3944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.611441 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.614532 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615176 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615297 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.621827 (+ 6530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:37.621836 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:37.621841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:37.621887 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:37.626066 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.627405 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.627413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.627419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.627572 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.628062 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.628068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.628071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.628430 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:37.628557 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:37.629728 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0809 05:35:37.629981 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:35:37.630395 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.588022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:49363 (local address 127.30.137.253:41497)
0809 05:35:37.588634 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:37.588639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:37.602616 (+ 13977us) server_negotiation.cc:407] Connection header received
0809 05:35:37.606129 (+ 3513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:37.606135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:37.606231 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:37.606311 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:37.610253 (+ 3942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.611414 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.614710 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615354 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615454 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.621879 (+ 6425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:37.621887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:37.621891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:37.621936 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:37.626066 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.627409 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.627415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.627418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.627582 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.628094 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.628099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.628103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.628457 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:37.628585 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:37.629763 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0809 05:35:37.629993 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":326,"threads_started":1}
I20260809 05:35:37.630427 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.590563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:36871 (local address 127.30.137.254:46411)
0809 05:35:37.604334 (+ 13771us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:37.604341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:37.604363 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:35:37.606259 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:37.606269 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:37.606375 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:37.606504 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:37.610295 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.611538 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.614542 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615242 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615345 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.621899 (+ 6554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:37.621905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:37.621910 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:37.621953 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:37.626377 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.627439 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.627444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.627447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.627595 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.628094 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.628099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.628103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.628466 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:37.628606 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:37.629744 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0809 05:35:37.629993 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13598,"thread_start_us":13800,"threads_started":1}
I20260809 05:35:37.632745 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.604433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.30.137.193:45463)
0809 05:35:37.605128 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:37.605153 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:37.605760 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:37.606862 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:37.606876 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:37.608685 (+ 1809us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:37.610033 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.610053 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.611656 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.611661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.614348 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.614370 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615661 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.616075 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.617160 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:37.617207 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:37.621534 (+ 4327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:37.626269 (+ 4735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:37.626284 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:37.626304 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:37.627251 (+ 947us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:37.627711 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:37.627715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:37.627719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:37.627945 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:37.628666 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:37.628676 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:37.629274 (+ 598us) client_negotiation.cc:769] Sending connection context
0809 05:35:37.630711 (+ 1437us) client_negotiation.cc:240] Negotiation successful
0809 05:35:37.631285 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":464,"spinlock_wait_cycles":37632,"thread_start_us":343,"threads_started":1}
I20260809 05:35:37.632790 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.588710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.30.137.193:49363)
0809 05:35:37.602300 (+ 13590us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:37.602385 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:37.605760 (+ 3375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:37.606862 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:37.606876 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:37.608687 (+ 1811us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:37.610024 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.610053 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.611656 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.611661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.614515 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.614526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615789 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615799 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.616070 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.617160 (+ 1090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:37.617198 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:37.621534 (+ 4336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:37.626285 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:37.626290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:37.626305 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:37.627250 (+ 945us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:37.627746 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:37.627751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:37.627754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:37.627955 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:37.628687 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:37.628692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:37.629287 (+ 595us) client_negotiation.cc:769] Sending connection context
0809 05:35:37.630615 (+ 1328us) client_negotiation.cc:240] Negotiation successful
0809 05:35:37.631839 (+ 1224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13046,"mutex_wait_us":1283,"spinlock_wait_cycles":81280,"thread_start_us":13857,"threads_started":1}
I20260809 05:35:37.633466 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.589634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:36871)
0809 05:35:37.603158 (+ 13524us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:37.603183 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:37.605761 (+ 2578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:37.606863 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:37.606879 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:37.608683 (+ 1804us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:37.610032 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.610059 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.611793 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.611798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.614361 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:37.614378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615684 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:37.616062 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.617433 (+ 1371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:37.617459 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:37.621678 (+ 4219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:37.626534 (+ 4856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:37.626539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:37.626543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:37.627295 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:37.627743 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:37.627748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:37.627751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:37.627955 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:37.628687 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:37.628692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:37.629287 (+ 595us) client_negotiation.cc:769] Sending connection context
0809 05:35:37.630615 (+ 1328us) client_negotiation.cc:240] Negotiation successful
0809 05:35:37.631452 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13311,"mutex_wait_us":830,"spinlock_wait_cycles":48128,"thread_start_us":13029,"threads_started":1}
I20260809 05:35:37.634596 9847 heartbeater.cc:347] Connected to a master server at 127.30.137.252:41165
I20260809 05:35:37.634932 9847 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:37.635728 9847 heartbeater.cc:511] Master 127.30.137.252:41165 requested a full tablet report, sending...
I20260809 05:35:37.636605 9848 heartbeater.cc:347] Connected to a master server at 127.30.137.254:46411
I20260809 05:35:37.636803 9848 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:37.637252 9848 heartbeater.cc:511] Master 127.30.137.254:46411 requested a full tablet report, sending...
I20260809 05:35:37.638227 9850 heartbeater.cc:347] Connected to a master server at 127.30.137.253:41497
I20260809 05:35:37.638376 9850 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:37.638867 9850 heartbeater.cc:511] Master 127.30.137.253:41497 requested a full tablet report, sending...
I20260809 05:35:37.639418 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:37.590163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.30.137.194:35907)
0809 05:35:37.607667 (+ 17504us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:37.607678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:37.608539 (+ 861us) server_negotiation.cc:407] Connection header received
0809 05:35:37.608625 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:37.608633 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:37.608956 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:37.609116 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:37.611464 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.613008 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.614866 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:37.615805 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:37.615933 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:37.622907 (+ 6974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:37.622926 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:37.622947 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:37.623008 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:37.634937 (+ 11929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.635942 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.635954 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.635958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.636141 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:37.636609 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:37.636615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:37.636621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:37.637097 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:37.637444 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:37.638598 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0809 05:35:37.638981 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17299,"thread_start_us":17097,"threads_started":1}
I20260809 05:35:37.639489 9568 ts_manager.cc:194] Registered new tserver with Master: 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:37.640611 8312 ts_manager.cc:194] Re-registered known tserver with Master: 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:37.640879 8822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:37.642288 8312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:36871
I20260809 05:35:37.644235 9276 ts_manager.cc:194] Registered new tserver with Master: 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:37.727712 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 8627
I20260809 05:35:37.753711 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:35907
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:46411
--webserver_port=45929
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:38.104882 9865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:38.105264 9865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:38.105441 9865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 05:35:38.107487 9662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: NON_VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: true } } }
I20260809 05:35:38.107933 9662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:38.112107 9662 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: attempting to promote NON_VOTER 62d9b52c7f974c8785e70bb8a805db42 to VOTER
I20260809 05:35:38.112887 9662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } }
W20260809 05:35:38.114327 9865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:38.114447 9865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:38.114634 9865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:38.115445 9583 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 LEARNER]: Refusing update from remote peer 5503b5dfd58e4712910fac17b2f1da82: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 05:35:38.116148 9295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Refusing update from remote peer 5503b5dfd58e4712910fac17b2f1da82: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 05:35:38.116068 9662 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 05:35:38.117007 9662 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 05:35:38.128085 9871 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.131517 9583 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.132273 9870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.132552 9870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:38.132817 9870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.133031 9870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:38.134397 9293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.134850 9865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:35907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=45929
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:38.137045 9865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:38.139638 9871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.140064 9865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 05:35:38.140206 9871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:38.141059 9882 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.141932 9882 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:35:38.143450 9879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5503b5dfd58e4712910fac17b2f1da82" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:38.143857 9879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 05:35:38.154928 9884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:38.155006 9885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:38.155465 9890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:38.160269 9865 server_base.cc:1034] running on GCE node
I20260809 05:35:38.161046 9865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:38.162057 9865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:38.163278 9865 hybrid_clock.cc:648] HybridClock initialized: now 1786253738163214 us; error 64 us; skew 500 ppm
I20260809 05:35:38.163565 9865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:38.166810 9865 init.cc:377] Logged in from keytab as oryx/127.30.137.194@KRBTEST.COM (short username oryx)
I20260809 05:35:38.168377 9865 webserver.cc:460] Webserver started at http://127.30.137.194:45929/ using document root <none> and password file <none>
I20260809 05:35:38.169163 9865 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:38.169320 9865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:38.174892 9865 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:38.177660 9901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:38.179047 9865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:38.179212 9865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "562c5edd49794b60b7fedb777ea228ab"
format_stamp: "Formatted at 2026-08-09 05:35:34 on dist-test-slave-3kk6"
server_key: "f5634aeb5ab73b399c60310ca7ef5be0"
server_key_iv: "e1a4a4eb2a9241fe35f9d42c70c3673d"
server_key_version: "encryptionkey@0"
I20260809 05:35:38.180285 9865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:38.196890 9865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:38.201226 9865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:38.201551 9865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:38.202579 9865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:38.204044 9865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:38.204167 9865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:38.204301 9865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:38.204391 9865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:38.239635 9865 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:35907
I20260809 05:35:38.239745 10073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:35907 every 8 connection(s)
I20260809 05:35:38.241160 9865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:38.251255 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 9865
I20260809 05:35:38.251612 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 8836
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.252@KRBTEST.COM
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:38.274511 10082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.244787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.30.137.194:59087)
0809 05:35:38.245905 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.245921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.249443 (+ 3522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.250984 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.250988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.251416 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:38.252502 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.252533 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.254873 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.254881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.258045 (+ 3164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.258066 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.259702 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.259712 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.260118 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.261016 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:38.261046 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:38.264147 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:38.269207 (+ 5060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.269216 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.269231 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.269886 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.270354 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.270359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.270362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.270605 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.271409 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:38.271418 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:38.271940 (+ 522us) client_negotiation.cc:769] Sending connection context
0809 05:35:38.272307 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 05:35:38.272885 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":251,"spinlock_wait_cycles":56704,"thread_start_us":533,"threads_started":1}
I20260809 05:35:38.274888 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.244159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:59087 (local address 127.30.137.253:41497)
0809 05:35:38.244733 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.244741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.246076 (+ 1335us) server_negotiation.cc:407] Connection header received
0809 05:35:38.250495 (+ 4419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.250504 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.250617 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.250762 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.252778 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.254589 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.258263 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.259143 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.259264 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.264372 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.264381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.264385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.264429 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.269077 (+ 4648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.270018 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.270024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.270028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.270212 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.270792 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.270800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.270806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.271256 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.271411 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.274188 (+ 2777us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.274422 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":276,"threads_started":1}
I20260809 05:35:38.276304 10076 heartbeater.cc:347] Connected to a master server at 127.30.137.253:41497
I20260809 05:35:38.276721 10076 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:38.277458 10080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.244168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:46053 (local address 127.30.137.254:46411)
0809 05:35:38.244980 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.244984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.246026 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 05:35:38.249800 (+ 3774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.249809 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.249947 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.250091 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.258819 (+ 8728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.260398 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.262263 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.263213 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.263366 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.268498 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.268506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.268511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.268564 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.272965 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.274327 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.274332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.274337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.274518 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.275509 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.275514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.275517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.275884 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.276027 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.276766 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.277044 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":647,"threads_started":1}
I20260809 05:35:38.277766 10076 heartbeater.cc:511] Master 127.30.137.253:41497 requested a full tablet report, sending...
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.195@KRBTEST.COM
I20260809 05:35:38.278563 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.244786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.194:46053)
0809 05:35:38.245824 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.245871 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.249450 (+ 3579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.250260 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.250269 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.251313 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:38.258616 (+ 7303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.258629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.260549 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.260555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.262098 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.262115 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.264334 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.264342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.264550 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.265133 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:38.265167 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:38.268295 (+ 3128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:38.273481 (+ 5186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.273487 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.273492 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.274203 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.275198 (+ 995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.275204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.275209 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.275378 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.276108 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:38.276114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:38.276318 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 05:35:38.277728 (+ 1410us) client_negotiation.cc:240] Negotiation successful
0809 05:35:38.278047 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":61,"spinlock_wait_cycles":9088,"thread_start_us":374,"threads_started":1}
I20260809 05:35:38.279372 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.246418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.30.137.194:56911)
0809 05:35:38.247088 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.247102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.249446 (+ 2344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.251747 (+ 2301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.251754 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.252473 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:38.252885 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.252902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.255008 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.255016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.258049 (+ 3033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.258071 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.261618 (+ 3547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.261628 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.261953 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.264091 (+ 2138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:38.264113 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:38.267790 (+ 3677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:38.273793 (+ 6003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.273798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.273801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.275616 (+ 1815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.276106 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.276113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.276117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.276304 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.277345 (+ 1041us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:38.277351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:38.277765 (+ 414us) client_negotiation.cc:769] Sending connection context
0809 05:35:38.278009 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 05:35:38.278306 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":150,"thread_start_us":389,"threads_started":1}
I20260809 05:35:38.279587 10075 heartbeater.cc:347] Connected to a master server at 127.30.137.254:46411
I20260809 05:35:38.279724 10075 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:38.279857 9276 ts_manager.cc:194] Registered new tserver with Master: 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907)
I20260809 05:35:38.279917 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.246592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:56911 (local address 127.30.137.252:41165)
0809 05:35:38.251125 (+ 4533us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.251131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.251150 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:35:38.251213 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.251217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.251324 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.251448 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.253134 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.254792 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.258286 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.261103 (+ 2817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.261262 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.268015 (+ 6753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.268022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.268027 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.268072 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.273631 (+ 5559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.275759 (+ 2128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.275766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.275771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.275951 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.276496 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.276502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.276506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.277105 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.277247 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.278959 (+ 1712us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.279196 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4388,"thread_start_us":6824,"threads_started":1}
I20260809 05:35:38.280182 10075 heartbeater.cc:511] Master 127.30.137.254:46411 requested a full tablet report, sending...
I20260809 05:35:38.280997 10074 heartbeater.cc:347] Connected to a master server at 127.30.137.252:41165
I20260809 05:35:38.281134 10074 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:38.281250 8312 ts_manager.cc:194] Re-registered known tserver with Master: 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907)
I20260809 05:35:38.281651 10074 heartbeater.cc:511] Master 127.30.137.252:41165 requested a full tablet report, sending...
I20260809 05:35:38.282529 8312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:46053
I20260809 05:35:38.282948 9568 ts_manager.cc:194] Registered new tserver with Master: 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907)
I20260809 05:35:38.287619 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.252684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.30.137.195:35527)
0809 05:35:38.263221 (+ 10537us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.263231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.263645 (+ 414us) server_negotiation.cc:407] Connection header received
0809 05:35:38.264524 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.264535 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.265107 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.265559 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.267947 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.269641 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.271732 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.272617 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.272756 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.280728 (+ 7972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.280768 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.280788 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.280844 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.284886 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.285544 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.285550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.285552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.285689 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.286008 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.286011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.286013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.286348 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.286596 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.286904 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.287194 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10324,"thread_start_us":10827,"threads_started":1}
I20260809 05:35:38.289017 9026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:38.372695 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 8836
I20260809 05:35:38.400830 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:35527
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:46411
--webserver_port=35847
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:35795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:35:38.647765 9848 heartbeater.cc:503] Master 127.30.137.254:46411 was elected leader, sending a full tablet report...
W20260809 05:35:38.753990 10090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:38.754271 10090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:38.754366 10090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:38.761876 10090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:38.761997 10090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:38.762115 10090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:38.772879 10090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:35795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:35527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=35847
--tserver_master_addrs=127.30.137.254:46411,127.30.137.253:41497,127.30.137.252:41165
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:38.774205 10090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:38.775583 10090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:38.789841 10103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:38.789923 10106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:38.789927 10104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:38.791013 10090 server_base.cc:1034] running on GCE node
I20260809 05:35:38.791610 10090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:38.792536 10090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:38.793730 10090 hybrid_clock.cc:648] HybridClock initialized: now 1786253738793674 us; error 63 us; skew 500 ppm
I20260809 05:35:38.794011 10090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:38.797255 10090 init.cc:377] Logged in from keytab as oryx/127.30.137.195@KRBTEST.COM (short username oryx)
I20260809 05:35:38.798907 10090 webserver.cc:460] Webserver started at http://127.30.137.195:35847/ using document root <none> and password file <none>
I20260809 05:35:38.799597 10090 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:38.799755 10090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:38.805250 10090 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 05:35:38.808106 10115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:38.809460 10090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:38.809636 10090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "abb2f48b70c44def90f83a95298d9215"
format_stamp: "Formatted at 2026-08-09 05:35:35 on dist-test-slave-3kk6"
server_key: "6bdd6d4fc26a01f10b03a4d01fb06779"
server_key_iv: "41eb5f0197b958adf27212e0bccb8dcf"
server_key_version: "encryptionkey@0"
I20260809 05:35:38.810694 10090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:38.828361 10090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:38.832347 10090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:38.832648 10090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:38.833640 10090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:38.835059 10090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:38.835147 10090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:38.835278 10090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:38.835381 10090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:38.868422 10090 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:35527
I20260809 05:35:38.868543 10298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:35527 every 8 connection(s)
I20260809 05:35:38.870054 10090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:38.879308 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 10090
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.253@KRBTEST.COM
I20260809 05:35:38.914355 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.873046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:33365 (local address 127.30.137.253:41497)
0809 05:35:38.888977 (+ 15931us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.888986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.889020 (+ 34us) server_negotiation.cc:407] Connection header received
0809 05:35:38.889828 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.889834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.889939 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.890058 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.892957 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.894513 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.898215 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.899118 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.899248 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.905733 (+ 6485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.905742 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.905748 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.905807 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.910019 (+ 4212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.911209 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.911215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.911219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.911394 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.911988 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.911994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.911998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.912370 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.912489 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.913623 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.913910 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15779,"thread_start_us":15979,"threads_started":1}
I20260809 05:35:38.917816 10305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.873839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.30.137.195:33365)
0809 05:35:38.886008 (+ 12169us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.886083 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.889411 (+ 3328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.890532 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.890545 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.891521 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:38.892651 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.892685 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.894860 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.894870 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.897971 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.897996 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.899730 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.899741 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.900177 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.901280 (+ 1103us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:38.901331 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:38.905386 (+ 4055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:38.910212 (+ 4826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.910222 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.910242 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.911060 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.911553 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.911558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.911562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.911853 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.912668 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:38.912681 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:38.913172 (+ 491us) client_negotiation.cc:769] Sending connection context
0809 05:35:38.915223 (+ 2051us) client_negotiation.cc:240] Negotiation successful
0809 05:35:38.915824 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11697,"thread_start_us":15892,"threads_started":1}
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:38.921792 10301 heartbeater.cc:347] Connected to a master server at 127.30.137.253:41497
I20260809 05:35:38.922190 10301 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:38.923723 10301 heartbeater.cc:511] Master 127.30.137.253:41497 requested a full tablet report, sending...
I20260809 05:35:38.925885 9276 ts_manager.cc:194] Registered new tserver with Master: abb2f48b70c44def90f83a95298d9215 (127.30.137.195:35527)
Aug 09 05:35:38 dist-test-slave-3kk6 krb5kdc[8241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253738, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.252@KRBTEST.COM
I20260809 05:35:38.931336 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.873839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:60907)
0809 05:35:38.903671 (+ 29832us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.903690 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.903808 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.905072 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.905078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.905784 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:38.906158 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.906172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.910590 (+ 4418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.910595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.912275 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.912291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.916421 (+ 4130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.916430 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.916638 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.917746 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:38.917776 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:38.921190 (+ 3414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:38.927967 (+ 6777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.927972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.927976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.928657 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.929137 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.929142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.929146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.929292 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.930102 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:38.930109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:38.930288 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 05:35:38.930430 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 05:35:38.930790 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29690,"mutex_wait_us":41,"spinlock_wait_cycles":52480,"thread_start_us":32402,"threads_started":1}
I20260809 05:35:38.932463 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.873044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:60907 (local address 127.30.137.254:46411)
0809 05:35:38.904518 (+ 31474us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.904526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.904547 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:35:38.904616 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.904624 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.904707 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.904844 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.909184 (+ 4340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.910395 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.915295 (+ 4900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.915958 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.916114 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.921919 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.921926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.921931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.921979 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.927769 (+ 5790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.928826 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.928834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.928837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.928985 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.929429 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.929434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.929437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.929869 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.930021 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.931857 (+ 1836us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.932100 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31340,"thread_start_us":31242,"threads_started":1}
I20260809 05:35:38.937800 10300 heartbeater.cc:347] Connected to a master server at 127.30.137.254:46411
I20260809 05:35:38.938014 10300 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:38.938642 10300 heartbeater.cc:511] Master 127.30.137.254:46411 requested a full tablet report, sending...
I20260809 05:35:38.940316 8312 ts_manager.cc:194] Re-registered known tserver with Master: abb2f48b70c44def90f83a95298d9215 (127.30.137.195:35527)
I20260809 05:35:38.941597 10306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.897523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.30.137.195:51755)
0809 05:35:38.906760 (+ 9237us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.906784 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.906919 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.907933 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.907941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.908576 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:38.908967 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.908982 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.913917 (+ 4935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.913922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.917464 (+ 3542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.917480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.919168 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.919175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:38.919435 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.923458 (+ 4023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:38.923485 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:38.928641 (+ 5156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:38.936344 (+ 7703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.936350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.936356 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.936876 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.937383 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:38.937389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:38.937392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:38.937576 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:38.940526 (+ 2950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:38.940531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:38.940709 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 05:35:38.940855 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 05:35:38.941167 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9071,"thread_start_us":6335,"threads_started":1}
I20260809 05:35:38.941808 8312 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:60907
I20260809 05:35:38.942724 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.891259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:51755 (local address 127.30.137.252:41165)
0809 05:35:38.903263 (+ 12004us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.903272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.907025 (+ 3753us) server_negotiation.cc:407] Connection header received
0809 05:35:38.907124 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.907131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.907253 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.907367 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.912495 (+ 5128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.913699 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.917949 (+ 4250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.918713 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.918834 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.928910 (+ 10076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.928920 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.928925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.928967 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.933401 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.937037 (+ 3636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.937043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.937047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.937237 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.937719 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.937724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.937727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.938318 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.938485 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.942072 (+ 3587us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.942335 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11819,"thread_start_us":12744,"threads_started":1}
I20260809 05:35:38.956807 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.910175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.30.137.254:46411)
0809 05:35:38.932731 (+ 22556us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.932737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.932758 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:35:38.932813 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.932819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.932936 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.933041 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.936255 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.937382 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.943989 (+ 6607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.944392 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.944530 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.948632 (+ 4102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.948640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.948647 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.948690 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.952048 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.953548 (+ 1500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.953555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.953558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.953714 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.954567 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.954572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.954574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.955016 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.955364 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.956187 (+ 823us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.956443 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22445,"threads_started":1}
I20260809 05:35:38.957226 10299 heartbeater.cc:347] Connected to a master server at 127.30.137.252:41165
I20260809 05:35:38.957435 10299 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:38.957768 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.910517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.30.137.253:41497)
0809 05:35:38.915149 (+ 4632us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.915155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.933935 (+ 18780us) server_negotiation.cc:407] Connection header received
0809 05:35:38.934233 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.934243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.934336 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.934415 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.938773 (+ 4358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.939909 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.941878 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.942500 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.942601 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.947747 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.947756 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.947762 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.947810 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.953354 (+ 5544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.954220 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.954226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.954230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.954404 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.955060 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.955068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.955072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.955458 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.955664 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.957203 (+ 1539us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.957467 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4511,"threads_started":1}
I20260809 05:35:38.958069 10299 heartbeater.cc:511] Master 127.30.137.252:41165 requested a full tablet report, sending...
I20260809 05:35:38.959892 9568 ts_manager.cc:194] Registered new tserver with Master: abb2f48b70c44def90f83a95298d9215 (127.30.137.195:35527)
I20260809 05:35:38.965448 8312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 05:35:38.975090 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.932168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.30.137.252:41165)
0809 05:35:38.943226 (+ 11058us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.943234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.956611 (+ 13377us) server_negotiation.cc:407] Connection header received
0809 05:35:38.958740 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.958749 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.958888 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.958995 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:38.962016 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.962979 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.964996 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:38.965727 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:38.965866 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:38.968511 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:38.968517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:38.968522 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:38.968565 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:38.972121 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.972879 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.972885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.972889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.973073 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:38.973416 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:38.973420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:38.973424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:38.973764 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:38.974020 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:38.974389 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 05:35:38.974637 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10916,"thread_start_us":24093,"threads_started":1}
W20260809 05:35:38.975919 8312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:39.005903 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.996591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.30.137.194:35907)
0809 05:35:38.997185 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.997199 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.997238 (+ 39us) server_negotiation.cc:407] Connection header received
0809 05:35:38.997371 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.997383 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.997835 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.998121 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:38.998989 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.000278 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.001993 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.003270 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.003466 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.003663 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.005469 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.005643 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":1167,"threads_started":1}
I20260809 05:35:39.005926 10342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.995736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:35907 (local address 127.0.0.1:39028)
0809 05:35:38.996818 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.996849 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.996982 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.998121 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.998129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.998166 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:38.998775 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:38.998791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.000406 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.000411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.001823 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.001836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.005069 (+ 3233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.005082 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.005280 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.005326 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 05:35:39.005419 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 05:35:39.005556 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":7655,"threads_started":1}
I20260809 05:35:39.007234 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.996095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:35527 (local address 127.0.0.1:33806)
0809 05:35:38.998876 (+ 2781us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:38.998911 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:38.999043 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:38.999748 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:38.999754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:38.999780 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:39.000148 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.000158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.002621 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.002627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.004789 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.004805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.006333 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.006339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.006536 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.006570 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 05:35:39.006703 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 05:35:39.006844 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2649,"mutex_wait_us":4,"thread_start_us":381,"threads_started":1}
I20260809 05:35:39.007282 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.996976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.30.137.195:35527)
0809 05:35:38.997699 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:38.997708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:38.999007 (+ 1299us) server_negotiation.cc:407] Connection header received
0809 05:35:38.999240 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:38.999251 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:38.999530 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:38.999747 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:39.000364 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.002431 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.004955 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.005934 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.006413 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.006592 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.006754 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.006928 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":905,"threads_started":1}
I20260809 05:35:39.009879 9969 tablet_service.cc:1467] Processing CreateTablet for tablet d25650af5c364b7ea00f56262e9f20d3 (DEFAULT_TABLE table=test-table [id=b3042e41bd9148998b6225ce65eb15eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:39.011274 9969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25650af5c364b7ea00f56262e9f20d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:39.010965 10189 tablet_service.cc:1467] Processing CreateTablet for tablet d25650af5c364b7ea00f56262e9f20d3 (DEFAULT_TABLE table=test-table [id=b3042e41bd9148998b6225ce65eb15eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:39.012521 10189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25650af5c364b7ea00f56262e9f20d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:39.013278 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.996849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:41035 (local address 127.0.0.1:39554)
0809 05:35:39.003720 (+ 6871us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:39.003741 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:39.003873 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:39.004657 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:39.004663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:39.004697 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:39.005186 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.005197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.008061 (+ 2864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.008068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.010093 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.010109 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.012371 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.012377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.012568 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.012604 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 05:35:39.012708 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 05:35:39.012855 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6743,"mutex_wait_us":4,"thread_start_us":1713,"threads_started":1}
I20260809 05:35:39.013787 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:38.997574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.30.137.193:41035)
0809 05:35:39.003158 (+ 5584us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.003171 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.003996 (+ 825us) server_negotiation.cc:407] Connection header received
0809 05:35:39.004083 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.004089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.004358 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.004671 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:39.006158 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.007930 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.010297 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.011802 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.011998 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.012181 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.012848 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.013049 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5390,"thread_start_us":5263,"threads_started":1}
I20260809 05:35:39.017571 9739 tablet_service.cc:1467] Processing CreateTablet for tablet d25650af5c364b7ea00f56262e9f20d3 (DEFAULT_TABLE table=test-table [id=b3042e41bd9148998b6225ce65eb15eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:39.019043 9739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d25650af5c364b7ea00f56262e9f20d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:39.025630 10347 tablet_bootstrap.cc:492] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Bootstrap starting.
I20260809 05:35:39.028386 10348 tablet_bootstrap.cc:492] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Bootstrap starting.
I20260809 05:35:39.030616 10347 tablet_bootstrap.cc:654] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:39.031941 10348 tablet_bootstrap.cc:654] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:39.032013 10347 log.cc:824] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:39.033216 10348 log.cc:824] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:39.035050 10352 tablet_bootstrap.cc:492] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Bootstrap starting.
I20260809 05:35:39.035565 10347 tablet_bootstrap.cc:492] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: No bootstrap required, opened a new log
I20260809 05:35:39.035851 10347 ts_tablet_manager.cc:1397] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 05:35:39.036928 10348 tablet_bootstrap.cc:492] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: No bootstrap required, opened a new log
I20260809 05:35:39.037206 10348 ts_tablet_manager.cc:1397] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 05:35:39.038084 10352 tablet_bootstrap.cc:654] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:39.039280 10352 log.cc:824] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:39.041714 10352 tablet_bootstrap.cc:492] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: No bootstrap required, opened a new log
I20260809 05:35:39.041986 10352 ts_tablet_manager.cc:1397] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 05:35:39.047000 10347 raft_consensus.cc:348] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.047458 10347 raft_consensus.cc:374] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:39.047569 10347 raft_consensus.cc:729] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb2f48b70c44def90f83a95298d9215, State: Initialized, Role: FOLLOWER
I20260809 05:35:39.048018 10348 raft_consensus.cc:348] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.048480 10348 raft_consensus.cc:374] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:39.048316 10347 consensus_queue.cc:260] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.048585 10348 raft_consensus.cc:729] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 562c5edd49794b60b7fedb777ea228ab, State: Initialized, Role: FOLLOWER
I20260809 05:35:39.049340 10348 consensus_queue.cc:260] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.050227 10352 raft_consensus.cc:348] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.050623 10352 raft_consensus.cc:374] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:39.050750 10352 raft_consensus.cc:729] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938f3ee41234415bb7818fd76ff7b626, State: Initialized, Role: FOLLOWER
I20260809 05:35:39.051102 10300 heartbeater.cc:503] Master 127.30.137.254:46411 was elected leader, sending a full tablet report...
I20260809 05:35:39.051432 10352 consensus_queue.cc:260] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.057296 10347 ts_tablet_manager.cc:1428] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Time spent starting tablet: real 0.021s user 0.014s sys 0.000s
I20260809 05:35:39.057296 10348 ts_tablet_manager.cc:1428] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Time spent starting tablet: real 0.020s user 0.013s sys 0.007s
I20260809 05:35:39.058282 10075 heartbeater.cc:503] Master 127.30.137.254:46411 was elected leader, sending a full tablet report...
I20260809 05:35:39.062014 10352 ts_tablet_manager.cc:1428] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Time spent starting tablet: real 0.020s user 0.016s sys 0.000s
W20260809 05:35:39.088879 9851 tablet.cc:2369] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:35:39.122610 10302 tablet.cc:2369] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:39.204146 10357 raft_consensus.cc:482] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:39.204553 10357 raft_consensus.cc:504] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.206454 10357 leader_election.cc:288] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907), 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:39.212870 10306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.207080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:35907 (local address 127.30.137.195:54233)
0809 05:35:39.207270 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:39.207297 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:39.207463 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:39.207889 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:39.207894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:39.207922 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:39.208170 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.208183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.209294 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.209299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.210721 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.210740 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.211990 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.211996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.212288 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.212316 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 05:35:39.212399 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 05:35:39.212551 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 05:35:39.212895 10343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.207124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:54233 (local address 127.30.137.194:35907)
0809 05:35:39.207339 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.207345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.207370 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:35:39.207614 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.207618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.207729 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.207841 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:39.208316 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.209143 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.210904 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.211807 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.211940 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.211992 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.212466 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.212612 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:35:39.212919 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.207106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:43005 (local address 127.30.137.193:41035)
0809 05:35:39.207346 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.207353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.207904 (+ 551us) server_negotiation.cc:407] Connection header received
0809 05:35:39.208034 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.208039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.208117 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.208219 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:39.208605 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.209343 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.211118 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.211970 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.212122 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.212203 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.212550 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.212684 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 05:35:39.212955 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.207080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:41035 (local address 127.30.137.195:43005)
0809 05:35:39.207786 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:39.207805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:39.207918 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:39.208256 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:39.208263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:39.208289 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:39.208468 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.208478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.209500 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.209504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.210970 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:39.210985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.212141 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.212146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:39.212370 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.212429 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 05:35:39.212523 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 05:35:39.212653 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":498}
I20260809 05:35:39.214444 9771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25650af5c364b7ea00f56262e9f20d3" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938f3ee41234415bb7818fd76ff7b626" is_pre_election: true
I20260809 05:35:39.214444 10003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25650af5c364b7ea00f56262e9f20d3" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "562c5edd49794b60b7fedb777ea228ab" is_pre_election: true
I20260809 05:35:39.215092 10003 raft_consensus.cc:2454] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 0.
I20260809 05:35:39.215091 9771 raft_consensus.cc:2454] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 0.
I20260809 05:35:39.215867 10118 leader_election.cc:302] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 938f3ee41234415bb7818fd76ff7b626, abb2f48b70c44def90f83a95298d9215; no voters:
I20260809 05:35:39.216212 10357 raft_consensus.cc:2781] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:39.216382 10357 raft_consensus.cc:482] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:39.216500 10357 raft_consensus.cc:3037] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:39.218894 10357 raft_consensus.cc:504] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.219558 10357 leader_election.cc:288] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 election: Requested vote from peers 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907), 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:39.220027 10003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25650af5c364b7ea00f56262e9f20d3" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "562c5edd49794b60b7fedb777ea228ab"
I20260809 05:35:39.220096 9771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d25650af5c364b7ea00f56262e9f20d3" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938f3ee41234415bb7818fd76ff7b626"
I20260809 05:35:39.220266 10003 raft_consensus.cc:3037] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:39.220335 9771 raft_consensus.cc:3037] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:39.222594 9771 raft_consensus.cc:2454] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 1.
I20260809 05:35:39.222592 10003 raft_consensus.cc:2454] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 1.
I20260809 05:35:39.223106 10118 leader_election.cc:302] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 938f3ee41234415bb7818fd76ff7b626, abb2f48b70c44def90f83a95298d9215; no voters:
I20260809 05:35:39.223419 10357 raft_consensus.cc:2781] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:39.223909 10357 raft_consensus.cc:686] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 1 LEADER]: Becoming Leader. State: Replica: abb2f48b70c44def90f83a95298d9215, State: Running, Role: LEADER
I20260809 05:35:39.224432 10357 consensus_queue.cc:237] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } }
I20260809 05:35:39.228955 8313 catalog_manager.cc:5495] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 reported cstate change: term changed from 0 to 1, leader changed from <none> to abb2f48b70c44def90f83a95298d9215 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "abb2f48b70c44def90f83a95298d9215" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } health_report { overall_health: UNKNOWN } } }
W20260809 05:35:39.244688 10077 tablet.cc:2369] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:39.262331 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.257207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.30.137.195:35527)
0809 05:35:39.257440 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.257448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.257578 (+ 130us) server_negotiation.cc:407] Connection header received
0809 05:35:39.257758 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.257762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.257845 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.257972 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:39.258292 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.259080 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.260358 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.260757 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.260850 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.261270 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:39.261765 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:39.261887 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.261950 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.262016 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:35:39.271030 10003 raft_consensus.cc:1261] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Refusing update from remote peer abb2f48b70c44def90f83a95298d9215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:39.271030 9771 raft_consensus.cc:1261] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Refusing update from remote peer abb2f48b70c44def90f83a95298d9215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:39.272225 10361 consensus_queue.cc:1035] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:39.272717 10357 consensus_queue.cc:1035] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:39.304057 8313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:39.304445 8313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:39.315519 8313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5503b5dfd58e4712910fac17b2f1da82: Sending DeleteTablet for 3 replicas of tablet d25650af5c364b7ea00f56262e9f20d3
I20260809 05:35:39.317457 9969 tablet_service.cc:1514] Processing DeleteTablet for tablet d25650af5c364b7ea00f56262e9f20d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:39 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:39028
I20260809 05:35:39.317606 10189 tablet_service.cc:1514] Processing DeleteTablet for tablet d25650af5c364b7ea00f56262e9f20d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:39 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:33806
I20260809 05:35:39.318760 9739 tablet_service.cc:1514] Processing DeleteTablet for tablet d25650af5c364b7ea00f56262e9f20d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:39 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:39554
I20260809 05:35:39.320238 10382 tablet_replica.cc:330] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: stopping tablet replica
I20260809 05:35:39.320760 10382 raft_consensus.cc:2229] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:39.321982 10380 tablet_replica.cc:330] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: stopping tablet replica
I20260809 05:35:39.322489 10380 raft_consensus.cc:2229] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:39.323132 10380 raft_consensus.cc:2258] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:39.328569 10382 raft_consensus.cc:2258] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:39.331481 10382 ts_tablet_manager.cc:1905] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:39.332091 10385 tablet_replica.cc:330] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: stopping tablet replica
I20260809 05:35:39.332610 10385 raft_consensus.cc:2229] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:39.333155 10385 raft_consensus.cc:2258] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:39.335080 10380 ts_tablet_manager.cc:1905] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:39.339623 10382 ts_tablet_manager.cc:1918] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:39.339915 10382 log.cc:1196] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/d25650af5c364b7ea00f56262e9f20d3
I20260809 05:35:39.340574 10382 ts_tablet_manager.cc:1939] T d25650af5c364b7ea00f56262e9f20d3 P abb2f48b70c44def90f83a95298d9215: Deleting consensus metadata
I20260809 05:35:39.342348 10380 ts_tablet_manager.cc:1918] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:39.342559 10380 log.cc:1196] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/d25650af5c364b7ea00f56262e9f20d3
I20260809 05:35:39.344034 10385 ts_tablet_manager.cc:1905] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:39.346912 8292 catalog_manager.cc:4841] TS abb2f48b70c44def90f83a95298d9215 (127.30.137.195:35527): tablet d25650af5c364b7ea00f56262e9f20d3 (table test-table [id=b3042e41bd9148998b6225ce65eb15eb]) successfully deleted
I20260809 05:35:39.347422 10380 ts_tablet_manager.cc:1939] T d25650af5c364b7ea00f56262e9f20d3 P 562c5edd49794b60b7fedb777ea228ab: Deleting consensus metadata
I20260809 05:35:39.349690 8295 catalog_manager.cc:4841] TS 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907): tablet d25650af5c364b7ea00f56262e9f20d3 (table test-table [id=b3042e41bd9148998b6225ce65eb15eb]) successfully deleted
I20260809 05:35:39.350493 10385 ts_tablet_manager.cc:1918] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:39.350968 10385 log.cc:1196] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/d25650af5c364b7ea00f56262e9f20d3
I20260809 05:35:39.351637 10385 ts_tablet_manager.cc:1939] T d25650af5c364b7ea00f56262e9f20d3 P 938f3ee41234415bb7818fd76ff7b626: Deleting consensus metadata
I20260809 05:35:39.352016 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 8265
I20260809 05:35:39.353266 8357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:39.353566 8292 catalog_manager.cc:4841] TS 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035): tablet d25650af5c364b7ea00f56262e9f20d3 (table test-table [id=b3042e41bd9148998b6225ce65eb15eb]) successfully deleted
I20260809 05:35:39.370509 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.319028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35134 (local address 127.30.137.253:41497)
0809 05:35:39.319276 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.319285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.340590 (+ 21305us) server_negotiation.cc:407] Connection header received
0809 05:35:39.340929 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.340936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.341022 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.341118 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:39.343703 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.345046 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.353040 (+ 7994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.353828 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.353951 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.362641 (+ 8690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:39.362680 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:39.362686 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:39.362743 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:39.367021 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.368075 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.368082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.368085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.368263 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.368759 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.368764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.368767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.369174 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:39.369452 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.369875 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.370128 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:35:39.389215 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.350782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.30.137.252:41165)
0809 05:35:39.350993 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.351000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.370800 (+ 19800us) server_negotiation.cc:407] Connection header received
0809 05:35:39.371093 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.371100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.371199 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.371285 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:39.373717 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.374862 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.376812 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.377494 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.377632 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.381415 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:39.381423 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:39.381428 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:39.381508 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:39.385924 (+ 4416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.386922 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.386928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.386932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.387112 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.387547 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.387553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.387556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.387947 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:39.388199 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.388582 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.388829 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:35:39.488137 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 8265
I20260809 05:35:39.545689 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.529478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.30.137.252:41165)
0809 05:35:39.529693 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.529701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.529842 (+ 141us) server_negotiation.cc:407] Connection header received
0809 05:35:39.530052 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.530057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.530131 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.530242 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:39.532363 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.533485 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.534896 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.535341 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.535484 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.538877 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:39.538883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:39.538886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:39.538920 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:39.542382 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.543308 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.543315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.543321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.543468 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.544133 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.544137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.544139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.544526 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:39.544795 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.545147 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.545331 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 05:35:39.547322 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:39.527283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.30.137.253:41497)
0809 05:35:39.527491 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:39.527498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:39.528649 (+ 1151us) server_negotiation.cc:407] Connection header received
0809 05:35:39.528995 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:39.529001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:39.529093 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:39.529172 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:39.531750 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.532748 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.535526 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:39.536120 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:39.536241 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:39.539441 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:39.539454 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:39.539457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:39.539501 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:39.543098 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.543872 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.543883 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.543885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.544008 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:39.544406 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:39.544411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:39.544415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:39.544776 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:39.544986 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:39.546247 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0809 05:35:39.546638 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:35:40.301995 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.300994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.194:42227)
0809 05:35:40.301588 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.301751 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":296,"threads_started":1}
W20260809 05:35:40.302507 10075 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:46411 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.320137 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.319186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:58925)
0809 05:35:40.319720 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.319899 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":255,"threads_started":1}
W20260809 05:35:40.320641 9848 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:46411 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.321646 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.320829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:37195)
0809 05:35:40.321324 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.321462 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":267,"threads_started":1}
W20260809 05:35:40.322064 10300 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:46411 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.822369 10400 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5503b5dfd58e4712910fac17b2f1da82)
I20260809 05:35:40.822700 10400 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } }
I20260809 05:35:40.823496 10401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5503b5dfd58e4712910fac17b2f1da82)
I20260809 05:35:40.823889 10400 leader_election.cc:288] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411), 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252:41165)
I20260809 05:35:40.823791 10401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } }
I20260809 05:35:40.825791 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.824056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48500)
0809 05:35:40.825129 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.825411 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":800,"threads_started":1}
W20260809 05:35:40.826236 9248 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.831996 10401 leader_election.cc:288] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411), a73393a6b7f24427981fb718086a93d3 (127.30.137.253:41497)
I20260809 05:35:40.840636 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.826104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48502)
0809 05:35:40.839980 (+ 13876us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.840275 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13734,"threads_started":1}
W20260809 05:35:40.840966 9537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.842211 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.841462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48524)
0809 05:35:40.841692 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.841923 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 05:35:40.842473 9248 leader_election.cc:334] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411): Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.843504 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.842875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48538)
0809 05:35:40.843097 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.843326 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 05:35:40.843688 9537 leader_election.cc:334] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411): Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.847721 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.824465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.30.137.252:41165)
0809 05:35:40.837927 (+ 13462us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:40.837936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:40.837965 (+ 29us) server_negotiation.cc:407] Connection header received
0809 05:35:40.838062 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:40.838070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:40.838205 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:40.840177 (+ 1972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:40.840460 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.843177 (+ 2717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.845281 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.846583 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.846749 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.846800 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:40.847316 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 05:35:40.847419 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13271,"thread_start_us":13878,"threads_started":1}
I20260809 05:35:40.847808 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.824170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:41165 (local address 127.0.0.1:34426)
0809 05:35:40.831364 (+ 7194us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.831393 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:40.831601 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:40.839523 (+ 7922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:40.839533 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:40.839578 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:40.839915 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.839937 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.843374 (+ 3437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.843382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.845121 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.845134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.846819 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.846831 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.847023 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.847151 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 05:35:40.847293 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 05:35:40.847443 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7041,"mutex_wait_us":74,"thread_start_us":16114,"threads_started":1}
I20260809 05:35:40.848588 9583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a73393a6b7f24427981fb718086a93d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "62d9b52c7f974c8785e70bb8a805db42" is_pre_election: true
I20260809 05:35:40.849005 9583 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a73393a6b7f24427981fb718086a93d3 in term 1.
I20260809 05:35:40.849211 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.838909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41497 (local address 127.0.0.1:35164)
0809 05:35:40.841530 (+ 2621us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.841552 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:40.841715 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:40.842451 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:40.842458 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:40.842494 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:40.842903 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.842919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.844612 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.844616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.846495 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.846516 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.848375 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.848381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.848549 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.848593 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 05:35:40.848704 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 05:35:40.848872 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2521,"thread_start_us":2401,"threads_started":1}
I20260809 05:35:40.849208 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.839911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.30.137.253:41497)
0809 05:35:40.841168 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:40.841174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:40.841849 (+ 675us) server_negotiation.cc:407] Connection header received
0809 05:35:40.841940 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:40.841945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:40.842023 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:40.842142 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:40.843117 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.844461 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.846769 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.848107 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.848291 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.848365 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:40.848730 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 05:35:40.848897 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":1180,"threads_started":1}
I20260809 05:35:40.849627 9252 leader_election.cc:302] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62d9b52c7f974c8785e70bb8a805db42, a73393a6b7f24427981fb718086a93d3; no voters: 5503b5dfd58e4712910fac17b2f1da82
I20260809 05:35:40.849972 10400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 05:35:40.850118 10400 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5503b5dfd58e4712910fac17b2f1da82)
I20260809 05:35:40.850262 10400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:35:40.850132 9293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "62d9b52c7f974c8785e70bb8a805db42" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a73393a6b7f24427981fb718086a93d3" is_pre_election: true
I20260809 05:35:40.852870 10400 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } }
I20260809 05:35:40.853338 9293 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 62d9b52c7f974c8785e70bb8a805db42 in current term 2: Already voted for candidate a73393a6b7f24427981fb718086a93d3 in this term.
I20260809 05:35:40.853654 10400 leader_election.cc:288] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 election: Requested vote from peers 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411), 62d9b52c7f974c8785e70bb8a805db42 (127.30.137.252:41165)
I20260809 05:35:40.854172 9583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a73393a6b7f24427981fb718086a93d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "62d9b52c7f974c8785e70bb8a805db42"
I20260809 05:35:40.854280 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.853771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48550)
0809 05:35:40.853987 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.854109 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260809 05:35:40.854444 9583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:35:40.854736 9539 leader_election.cc:302] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62d9b52c7f974c8785e70bb8a805db42; no voters: 5503b5dfd58e4712910fac17b2f1da82, a73393a6b7f24427981fb718086a93d3
I20260809 05:35:40.855472 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.855067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48566)
0809 05:35:40.855275 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.855354 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260809 05:35:40.855669 9248 leader_election.cc:334] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411): Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
I20260809 05:35:40.857084 9583 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a73393a6b7f24427981fb718086a93d3 in term 2.
I20260809 05:35:40.857290 10401 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 05:35:40.857599 9252 leader_election.cc:302] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62d9b52c7f974c8785e70bb8a805db42, a73393a6b7f24427981fb718086a93d3; no voters: 5503b5dfd58e4712910fac17b2f1da82
I20260809 05:35:40.857913 10400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 05:35:40.858417 10400 raft_consensus.cc:686] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [term 2 LEADER]: Becoming Leader. State: Replica: a73393a6b7f24427981fb718086a93d3, State: Running, Role: LEADER
I20260809 05:35:40.858952 10400 consensus_queue.cc:237] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } }
I20260809 05:35:40.861047 10413 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a73393a6b7f24427981fb718086a93d3. Latest consensus state: current_term: 2 leader_uuid: "a73393a6b7f24427981fb718086a93d3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:40.861532 10413 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:40.862260 10416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:40.865892 10416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:40.866828 10416 catalog_manager.cc:1259] Loaded cluster ID: e3949a3c8a6845f29237bf3225b0be8a
I20260809 05:35:40.866930 10416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:40.868114 10416 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:40.868956 10416 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Loaded TSK: 0
I20260809 05:35:40.869822 10416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:35:40.898002 9276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:40.906396 9276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:40.911019 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.910483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48576)
0809 05:35:40.910762 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.910848 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 05:35:40.911096 9583 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [term 2 FOLLOWER]: Refusing update from remote peer a73393a6b7f24427981fb718086a93d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 05:35:40.911883 10413 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 05:35:40.912178 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.911759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48586)
0809 05:35:40.911934 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.912018 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260809 05:35:40.912418 9248 consensus_peers.cc:487] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 -> Peer 5503b5dfd58e4712910fac17b2f1da82 (127.30.137.254:46411): Couldn't send request to peer 5503b5dfd58e4712910fac17b2f1da82. Status: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 05:35:40.914976 10401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a73393a6b7f24427981fb718086a93d3. Latest consensus state: current_term: 2 leader_uuid: "a73393a6b7f24427981fb718086a93d3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:40.915273 10401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:35:40.917011 10413 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a73393a6b7f24427981fb718086a93d3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:40.917375 10413 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:40.917913 10401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a73393a6b7f24427981fb718086a93d3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "5503b5dfd58e4712910fac17b2f1da82" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46411 } } peers { permanent_uuid: "a73393a6b7f24427981fb718086a93d3" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41497 } attrs { promote: false } } peers { permanent_uuid: "62d9b52c7f974c8785e70bb8a805db42" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 41165 } attrs { promote: false } } }
I20260809 05:35:40.918155 10401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62d9b52c7f974c8785e70bb8a805db42 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:35:40.918769 10419 mvcc.cc:205] Tried to move back new op lower bound from 7316495322764091392 to 7316495322562039808. Current Snapshot: MvccSnapshot[applied={T|T < 7316495322764091392 or (T in {7316495322764091392})}]
I20260809 05:35:40.936601 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.927146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:35527 (local address 127.0.0.1:33816)
0809 05:35:40.927934 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.927956 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:40.928087 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:40.928767 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:40.928777 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:40.928815 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:40.929388 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.929404 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.931413 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.931423 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.933799 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.933815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.935614 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.935620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.935809 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.935846 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:35:40.935951 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 05:35:40.936104 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":4,"thread_start_us":384,"threads_started":1}
I20260809 05:35:40.936523 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.927304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.30.137.195:35527)
0809 05:35:40.928123 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:40.928132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:40.928160 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:35:40.928275 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:40.928283 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:40.928385 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:40.928516 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:40.929643 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.931157 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.933991 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.935367 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.935545 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.935625 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:40.936004 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 05:35:40.936175 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":516,"threads_started":1}
I20260809 05:35:40.937011 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.926275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:35907 (local address 127.0.0.1:39036)
0809 05:35:40.926464 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.926487 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:40.926617 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:40.931413 (+ 4796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:40.931418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:40.931447 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:40.931715 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.931727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.933226 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.933231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.934898 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.934910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.936278 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.936284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.936458 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.936494 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 05:35:40.936588 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 05:35:40.936740 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 05:35:40.937716 10189 tablet_service.cc:1467] Processing CreateTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba (DEFAULT_TABLE table=test-table [id=3bdc7ae210db4cea80473cf941aba514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:40.937886 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.926400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.30.137.194:35907)
0809 05:35:40.931104 (+ 4704us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:40.931109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:40.931127 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:35:40.931190 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:40.931194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:40.931254 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:40.931382 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:40.931934 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.933053 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.935049 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.936087 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.937263 (+ 1176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.937348 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:40.937402 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 05:35:40.937537 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4593,"thread_start_us":4475,"threads_started":1}
I20260809 05:35:40.938271 10189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722b9ca28d584b3d94efa0f6a4fa2eba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:40.938596 9969 tablet_service.cc:1467] Processing CreateTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba (DEFAULT_TABLE table=test-table [id=3bdc7ae210db4cea80473cf941aba514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:40.939034 9969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722b9ca28d584b3d94efa0f6a4fa2eba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:40.939546 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.926834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:41035 (local address 127.0.0.1:39568)
0809 05:35:40.927043 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.927063 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:40.927196 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:40.932292 (+ 5096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:40.932298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:40.932329 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:40.932722 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.932735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.934070 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.934075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.936053 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:40.936068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.938771 (+ 2703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.938777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:40.938971 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.939016 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 05:35:40.939090 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 05:35:40.939239 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":31}
I20260809 05:35:40.940677 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.927632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.30.137.193:41035)
0809 05:35:40.931922 (+ 4290us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:40.931928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:40.931945 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:35:40.932018 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:40.932025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:40.932125 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:40.932663 (+ 538us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:40.932850 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.933948 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.936269 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:40.938493 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:40.940080 (+ 1587us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:40.940162 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:40.940222 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 05:35:40.940378 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4145,"thread_start_us":4044,"threads_started":1}
I20260809 05:35:40.941607 9739 tablet_service.cc:1467] Processing CreateTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba (DEFAULT_TABLE table=test-table [id=3bdc7ae210db4cea80473cf941aba514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:40.942059 9739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 722b9ca28d584b3d94efa0f6a4fa2eba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:40.944202 10435 tablet_bootstrap.cc:492] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Bootstrap starting.
I20260809 05:35:40.946029 10435 tablet_bootstrap.cc:654] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:40.947861 10436 tablet_bootstrap.cc:492] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Bootstrap starting.
I20260809 05:35:40.948664 10435 tablet_bootstrap.cc:492] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: No bootstrap required, opened a new log
I20260809 05:35:40.948835 10435 ts_tablet_manager.cc:1397] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 05:35:40.949597 10439 tablet_bootstrap.cc:492] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Bootstrap starting.
I20260809 05:35:40.949692 10435 raft_consensus.cc:348] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.950002 10436 tablet_bootstrap.cc:654] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:40.949994 10435 raft_consensus.cc:374] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:40.950094 10435 raft_consensus.cc:729] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 562c5edd49794b60b7fedb777ea228ab, State: Initialized, Role: FOLLOWER
I20260809 05:35:40.950374 10435 consensus_queue.cc:260] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.951427 10435 ts_tablet_manager.cc:1428] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 05:35:40.959285 10439 tablet_bootstrap.cc:654] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:40.960785 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.952813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.194:59373)
0809 05:35:40.960153 (+ 7340us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.960413 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7128,"thread_start_us":6892,"threads_started":1}
I20260809 05:35:40.961959 10439 tablet_bootstrap.cc:492] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: No bootstrap required, opened a new log
I20260809 05:35:40.962128 10439 ts_tablet_manager.cc:1397] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260809 05:35:40.962443 10436 tablet_bootstrap.cc:492] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: No bootstrap required, opened a new log
I20260809 05:35:40.962642 10436 ts_tablet_manager.cc:1397] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.012s
I20260809 05:35:40.963126 10439 raft_consensus.cc:348] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.963392 10439 raft_consensus.cc:374] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:40.963510 10439 raft_consensus.cc:729] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938f3ee41234415bb7818fd76ff7b626, State: Initialized, Role: FOLLOWER
I20260809 05:35:40.963445 10436 raft_consensus.cc:348] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.963707 10436 raft_consensus.cc:374] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:40.963817 10436 raft_consensus.cc:729] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb2f48b70c44def90f83a95298d9215, State: Initialized, Role: FOLLOWER
I20260809 05:35:40.963826 10439 consensus_queue.cc:260] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.964068 10436 consensus_queue.cc:260] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.964749 10439 ts_tablet_manager.cc:1428] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 05:35:40.964929 10436 ts_tablet_manager.cc:1428] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 05:35:40.969448 10301 heartbeater.cc:503] Master 127.30.137.253:41497 was elected leader, sending a full tablet report...
I20260809 05:35:40.970289 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.965541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:59199)
0809 05:35:40.969802 (+ 4261us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.969962 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4150,"thread_start_us":3905,"threads_started":1}
I20260809 05:35:40.970896 9850 heartbeater.cc:503] Master 127.30.137.253:41497 was elected leader, sending a full tablet report...
I20260809 05:35:40.971835 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.965738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:59063)
0809 05:35:40.971344 (+ 5606us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.971533 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5406,"threads_started":1}
I20260809 05:35:40.972618 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.970966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:54317)
0809 05:35:40.972370 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.972460 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1301,"threads_started":1}
I20260809 05:35:40.974159 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:40.973818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:58737)
0809 05:35:40.973971 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:40.974050 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 05:35:40.988493 10451 raft_consensus.cc:482] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:40.988781 10451 raft_consensus.cc:504] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.989535 10451 leader_election.cc:288] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907), 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:40.990222 9771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722b9ca28d584b3d94efa0f6a4fa2eba" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938f3ee41234415bb7818fd76ff7b626" is_pre_election: true
I20260809 05:35:40.990250 10003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722b9ca28d584b3d94efa0f6a4fa2eba" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "562c5edd49794b60b7fedb777ea228ab" is_pre_election: true
I20260809 05:35:40.990490 9771 raft_consensus.cc:2454] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 0.
I20260809 05:35:40.990549 10003 raft_consensus.cc:2454] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 0.
I20260809 05:35:40.991114 10118 leader_election.cc:302] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 938f3ee41234415bb7818fd76ff7b626, abb2f48b70c44def90f83a95298d9215; no voters:
I20260809 05:35:40.991436 10451 raft_consensus.cc:2781] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:40.991565 10451 raft_consensus.cc:482] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:40.991676 10451 raft_consensus.cc:3037] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:40.993916 10451 raft_consensus.cc:504] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:40.994580 10451 leader_election.cc:288] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 election: Requested vote from peers 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907), 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035)
I20260809 05:35:40.995297 9771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722b9ca28d584b3d94efa0f6a4fa2eba" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938f3ee41234415bb7818fd76ff7b626"
I20260809 05:35:40.995298 10003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "722b9ca28d584b3d94efa0f6a4fa2eba" candidate_uuid: "abb2f48b70c44def90f83a95298d9215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "562c5edd49794b60b7fedb777ea228ab"
I20260809 05:35:40.995551 10003 raft_consensus.cc:3037] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:40.995576 9771 raft_consensus.cc:3037] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:40.998769 10003 raft_consensus.cc:2454] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 1.
I20260809 05:35:40.998769 9771 raft_consensus.cc:2454] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb2f48b70c44def90f83a95298d9215 in term 1.
I20260809 05:35:40.999228 10118 leader_election.cc:302] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 938f3ee41234415bb7818fd76ff7b626, abb2f48b70c44def90f83a95298d9215; no voters:
I20260809 05:35:40.999557 10451 raft_consensus.cc:2781] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:41.000051 10451 raft_consensus.cc:686] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 1 LEADER]: Becoming Leader. State: Replica: abb2f48b70c44def90f83a95298d9215, State: Running, Role: LEADER
I20260809 05:35:41.000377 10451 consensus_queue.cc:237] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } }
I20260809 05:35:41.001626 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.000796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:59995)
0809 05:35:41.001042 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.001257 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 05:35:41.003518 9275 catalog_manager.cc:5495] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 reported cstate change: term changed from 0 to 1, leader changed from <none> to abb2f48b70c44def90f83a95298d9215 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "abb2f48b70c44def90f83a95298d9215" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abb2f48b70c44def90f83a95298d9215" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 35527 } health_report { overall_health: HEALTHY } } }
I20260809 05:35:41.024338 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.019835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.30.137.195:35527)
0809 05:35:41.020030 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:41.020034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:41.020056 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:35:41.020255 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:41.020258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:41.020326 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:41.020411 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:41.020736 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.021439 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.022773 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.023255 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.023349 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:41.023729 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:41.023823 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:41.023894 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:41.024026 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 05:35:41.024089 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:35:41.027753 9771 raft_consensus.cc:1261] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Refusing update from remote peer abb2f48b70c44def90f83a95298d9215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:41.027753 10003 raft_consensus.cc:1261] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Refusing update from remote peer abb2f48b70c44def90f83a95298d9215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:41.028522 10455 consensus_queue.cc:1035] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "938f3ee41234415bb7818fd76ff7b626" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 41035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:41.028877 10451 consensus_queue.cc:1035] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "562c5edd49794b60b7fedb777ea228ab" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 35907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:41.030823 10076 heartbeater.cc:503] Master 127.30.137.253:41497 was elected leader, sending a full tablet report...
I20260809 05:35:41.031767 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.031035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.194:36137)
0809 05:35:41.031395 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.031531 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260809 05:35:41.032310 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.031827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:48677)
0809 05:35:41.032011 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.032104 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 05:35:41.037838 10459 mvcc.cc:205] Tried to move back new op lower bound from 7316495323243020288 to 7316495323140321280. Current Snapshot: MvccSnapshot[applied={T|T < 7316495323243020288}]
I20260809 05:35:41.039660 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.039072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.194:38567)
0809 05:35:41.039349 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.039464 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 05:35:41.040293 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.039464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:34437)
0809 05:35:41.040026 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.040126 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446}
I20260809 05:35:41.045199 10460 mvcc.cc:205] Tried to move back new op lower bound from 7316495323243020288 to 7316495323140321280. Current Snapshot: MvccSnapshot[applied={T|T < 7316495323243020288}]
I20260809 05:35:41.046319 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.045873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.195:60989)
0809 05:35:41.046020 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.046116 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 05:35:41.046556 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.045928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.30.137.193:37123)
0809 05:35:41.046221 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.046340 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260809 05:35:41.048650 9276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:41.048938 9276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:41.055423 9276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a73393a6b7f24427981fb718086a93d3: Sending DeleteTablet for 3 replicas of tablet 722b9ca28d584b3d94efa0f6a4fa2eba
I20260809 05:35:41.056449 9969 tablet_service.cc:1514] Processing DeleteTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:41 UTC) from {username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:39036
I20260809 05:35:41.056619 10189 tablet_service.cc:1514] Processing DeleteTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:41 UTC) from {username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:33816
I20260809 05:35:41.057065 9739 tablet_service.cc:1514] Processing DeleteTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:41 UTC) from {username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:39568
I20260809 05:35:41.057545 9738 tablet_service.cc:1514] Processing DeleteTablet for tablet 722b9ca28d584b3d94efa0f6a4fa2eba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:41 UTC) from {username='oryx', principal='oryx/127.30.137.253@KRBTEST.COM'} at 127.0.0.1:39568
I20260809 05:35:41.057801 10467 tablet_replica.cc:330] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: stopping tablet replica
I20260809 05:35:41.058086 10467 raft_consensus.cc:2229] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:41.058605 10467 raft_consensus.cc:2258] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:41.059154 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 9544
I20260809 05:35:41.060129 10467 ts_tablet_manager.cc:1905] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:41.068135 10470 tablet_replica.cc:330] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: stopping tablet replica
I20260809 05:35:41.068416 10470 raft_consensus.cc:2229] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:41.068838 10470 raft_consensus.cc:2258] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:41.069607 10467 ts_tablet_manager.cc:1918] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:41.069809 10467 log.cc:1196] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/722b9ca28d584b3d94efa0f6a4fa2eba
I20260809 05:35:41.070117 10470 ts_tablet_manager.cc:1905] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:41.070400 10467 ts_tablet_manager.cc:1939] T 722b9ca28d584b3d94efa0f6a4fa2eba P abb2f48b70c44def90f83a95298d9215: Deleting consensus metadata
I20260809 05:35:41.072136 10466 tablet_replica.cc:330] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: stopping tablet replica
I20260809 05:35:41.072065 9251 catalog_manager.cc:4841] TS abb2f48b70c44def90f83a95298d9215 (127.30.137.195:35527): tablet 722b9ca28d584b3d94efa0f6a4fa2eba (table test-table [id=3bdc7ae210db4cea80473cf941aba514]) successfully deleted
I20260809 05:35:41.072345 10466 raft_consensus.cc:2229] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:41.072659 10466 raft_consensus.cc:2258] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:41.073674 10466 ts_tablet_manager.cc:1905] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:41.077962 10470 ts_tablet_manager.cc:1918] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:41.078163 10470 log.cc:1196] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/722b9ca28d584b3d94efa0f6a4fa2eba
I20260809 05:35:41.078749 10470 ts_tablet_manager.cc:1939] T 722b9ca28d584b3d94efa0f6a4fa2eba P 938f3ee41234415bb7818fd76ff7b626: Deleting consensus metadata
I20260809 05:35:41.079448 10466 ts_tablet_manager.cc:1918] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:41.079613 10466 log.cc:1196] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/722b9ca28d584b3d94efa0f6a4fa2eba
I20260809 05:35:41.080130 10466 ts_tablet_manager.cc:1939] T 722b9ca28d584b3d94efa0f6a4fa2eba P 562c5edd49794b60b7fedb777ea228ab: Deleting consensus metadata
I20260809 05:35:41.080201 9251 catalog_manager.cc:4841] TS 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035): tablet 722b9ca28d584b3d94efa0f6a4fa2eba (table test-table [id=3bdc7ae210db4cea80473cf941aba514]) successfully deleted
W20260809 05:35:41.080888 9251 catalog_manager.cc:4808] TS 938f3ee41234415bb7818fd76ff7b626 (127.30.137.193:41035): delete failed for tablet 722b9ca28d584b3d94efa0f6a4fa2eba because the tablet was not found. No further retry: Not found: Tablet not found: 722b9ca28d584b3d94efa0f6a4fa2eba
I20260809 05:35:41.081261 9256 catalog_manager.cc:4841] TS 562c5edd49794b60b7fedb777ea228ab (127.30.137.194:35907): tablet 722b9ca28d584b3d94efa0f6a4fa2eba (table test-table [id=3bdc7ae210db4cea80473cf941aba514]) successfully deleted
I20260809 05:35:41.088547 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.060033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.30.137.193:41035)
0809 05:35:41.060232 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:41.060239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:41.071597 (+ 11358us) server_negotiation.cc:407] Connection header received
0809 05:35:41.071862 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:41.071869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:41.071962 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:41.072079 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:41.074240 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.075375 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.077216 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.077905 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.078051 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:41.082183 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:41.082198 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:41.082213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:41.082250 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:41.085791 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:41.086521 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:41.086528 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:41.086530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:41.086777 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:41.087119 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:41.087122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:41.087124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:41.087500 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:41.087721 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:41.087992 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 05:35:41.088181 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:35:41.089248 9839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:41.190245 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 9544
I20260809 05:35:41.216339 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 9865
I20260809 05:35:41.230159 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.217205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.30.137.194:35907)
0809 05:35:41.217402 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:41.217406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:41.217425 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:35:41.217600 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:41.217604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:41.217670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:41.217758 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:41.219321 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.220021 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.221344 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.221726 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.221819 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:41.224802 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:41.224820 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:41.224837 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:41.224880 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:41.227744 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:41.228435 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:41.228442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:41.228444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:41.228579 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:41.228905 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:41.228908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:41.228910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:41.229248 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:41.229493 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:41.229712 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 05:35:41.229896 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:35:41.230973 10067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:41.287107 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.286524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48594)
0809 05:35:41.286773 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.286896 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 05:35:41.288220 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.287816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:46411 (local address 127.0.0.1:48600)
0809 05:35:41.287971 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:41.288060 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 05:35:41.327975 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 9865
I20260809 05:35:41.352130 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 10090
I20260809 05:35:41.367661 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:41.353030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.30.137.195:35527)
0809 05:35:41.353255 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:41.353261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:41.353286 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:35:41.353432 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:41.353438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:41.353535 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:41.353654 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:41.355347 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.356485 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.357733 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:41.358395 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:41.358543 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:41.361618 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:41.361638 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:41.361693 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:41.361732 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:41.364814 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:41.365599 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:41.365607 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:41.365610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:41.365774 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:41.366157 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:41.366161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:41.366163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:41.366577 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:41.366939 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:41.367212 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 05:35:41.367398 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:35:41.368438 10290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:41.493913 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 10090
I20260809 05:35:41.527052 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 9044
I20260809 05:35:41.528200 9320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:41.657945 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 9044
I20260809 05:35:41.692158 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 9216
I20260809 05:35:41.693377 9615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:41.816212 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 9216
2026-08-09T05:35:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10241 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:41 dist-test-slave-3kk6 krb5kdc[10480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:41 dist-test-slave-3kk6 krb5kdc[10480](info): set up 2 sockets
Aug 09 05:35:41 dist-test-slave-3kk6 krb5kdc[10480](info): commencing operation
krb5kdc: starting...
W20260809 05:35:43.140823 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:43 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab.
Entry for principal oryx/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab.
I20260809 05:35:43.421979 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44221
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab
--principal=oryx/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:44221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:43.775862 10502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:43.776178 10502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:43.776281 10502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:43.783633 10502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:43.783771 10502 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 05:35:43.783813 10502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:43.783878 10502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:43.783931 10502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:43.783987 10502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:43.818504 10502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:44221
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.254.keytab
--principal=oryx/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:43.820375 10502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:43.821795 10502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:43.834129 10509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:43.834187 10510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:43.834213 10514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:43.835103 10502 server_base.cc:1034] running on GCE node
I20260809 05:35:43.835930 10502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:43.837427 10502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:43.838763 10502 hybrid_clock.cc:648] HybridClock initialized: now 1786253743838644 us; error 34 us; skew 500 ppm
I20260809 05:35:43.839133 10502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:43 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253743, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:43.842511 10502 init.cc:377] Logged in from keytab as oryx/127.30.137.254@KRBTEST.COM (short username oryx)
I20260809 05:35:43.844178 10502 webserver.cc:460] Webserver started at http://127.30.137.254:35245/ using document root <none> and password file <none>
I20260809 05:35:43.844923 10502 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:43.845063 10502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:43.845402 10502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:43.848522 10502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "0a6238bd708d4cb789cf8159dff136cf"
format_stamp: "Formatted at 2026-08-09 05:35:43 on dist-test-slave-3kk6"
server_key: "7e8b85a539fbaa01f60a0463c5b4c784"
server_key_iv: "14be2db91ed00e51abaa136cc437ae1c"
server_key_version: "encryptionkey@0"
I20260809 05:35:43.849375 10502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "0a6238bd708d4cb789cf8159dff136cf"
format_stamp: "Formatted at 2026-08-09 05:35:43 on dist-test-slave-3kk6"
server_key: "7e8b85a539fbaa01f60a0463c5b4c784"
server_key_iv: "14be2db91ed00e51abaa136cc437ae1c"
server_key_version: "encryptionkey@0"
I20260809 05:35:43.854692 10502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 05:35:43.858115 10523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:43.859511 10502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:43.859735 10502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "0a6238bd708d4cb789cf8159dff136cf"
format_stamp: "Formatted at 2026-08-09 05:35:43 on dist-test-slave-3kk6"
server_key: "7e8b85a539fbaa01f60a0463c5b4c784"
server_key_iv: "14be2db91ed00e51abaa136cc437ae1c"
server_key_version: "encryptionkey@0"
I20260809 05:35:43.859973 10502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:43.878345 10502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:43.883144 10502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:43.883446 10502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:43.907107 10502 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:44221
I20260809 05:35:43.907115 10614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:44221 every 8 connection(s)
I20260809 05:35:43.908599 10502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:43.912529 10615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:43.913198 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 10502
I20260809 05:35:43.913430 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:43.914335 31271 external_mini_cluster.cc:1428] Setting key 54a1af8f13d1802bdc202e49ef9eedae
I20260809 05:35:43.927572 10615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Bootstrap starting.
Aug 09 05:35:43 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:43.931603 10615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:43.933056 10615 log.cc:824] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:43.936067 10615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: No bootstrap required, opened a new log
I20260809 05:35:43.939141 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:43.916650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.30.137.254:44221)
0809 05:35:43.917561 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:43.917573 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:43.917623 (+ 50us) server_negotiation.cc:407] Connection header received
0809 05:35:43.919462 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:43.919487 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:43.919921 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:43.920304 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:43.922230 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:43.923662 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:43.925477 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:43.926136 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:43.926265 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:43.931137 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:43.931148 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:43.931167 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:43.931213 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:43.934843 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:43.935617 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:43.935624 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:43.935630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:43.935817 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:43.936215 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:43.936223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:43.936226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:43.936755 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:43.937074 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:43.937554 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 05:35:43.938014 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":440,"threads_started":1}
I20260809 05:35:43.944262 10615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6238bd708d4cb789cf8159dff136cf" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44221 } }
I20260809 05:35:43.944682 10615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:43.944803 10615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a6238bd708d4cb789cf8159dff136cf, State: Initialized, Role: FOLLOWER
I20260809 05:35:43.945411 10615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6238bd708d4cb789cf8159dff136cf" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44221 } }
I20260809 05:35:43.945670 10615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:43.945818 10615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:43.945996 10615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:43.948750 10615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6238bd708d4cb789cf8159dff136cf" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44221 } }
I20260809 05:35:43.949281 10615 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a6238bd708d4cb789cf8159dff136cf; no voters:
I20260809 05:35:43.949888 10615 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:43.950026 10620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:43.950800 10620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [term 1 LEADER]: Becoming Leader. State: Replica: 0a6238bd708d4cb789cf8159dff136cf, State: Running, Role: LEADER
I20260809 05:35:43.951424 10620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6238bd708d4cb789cf8159dff136cf" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44221 } }
I20260809 05:35:43.952134 10615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:43.954177 10622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a6238bd708d4cb789cf8159dff136cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6238bd708d4cb789cf8159dff136cf" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44221 } } }
I20260809 05:35:43.954517 10623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6238bd708d4cb789cf8159dff136cf. Latest consensus state: current_term: 1 leader_uuid: "0a6238bd708d4cb789cf8159dff136cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6238bd708d4cb789cf8159dff136cf" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44221 } } }
I20260809 05:35:43.954622 10622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:43.954820 10623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:43.956837 10627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:43.962436 10627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:43.971309 10627 catalog_manager.cc:1347] Generated new cluster ID: a0a8dea161184ac084ff19b57093adc8
I20260809 05:35:43.971443 10627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:44.002907 10627 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:44.004372 10627 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:44.012439 10627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Generated new TSK 0
I20260809 05:35:44.013383 10627 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab.
Entry for principal oryx/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab.
I20260809 05:35:44.095274 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44221
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:44.446346 10651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:44.446643 10651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:44.446820 10651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:44.454342 10651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:44.454454 10651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:44.454538 10651 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:35:44.454620 10651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:44.464890 10651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:44221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:44.466198 10651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:44.467569 10651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:44.481166 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:44.481244 10659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:44.481140 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:44.481971 10651 server_base.cc:1034] running on GCE node
I20260809 05:35:44.482725 10651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:44.483731 10651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:44.484983 10651 hybrid_clock.cc:648] HybridClock initialized: now 1786253744484921 us; error 46 us; skew 500 ppm
I20260809 05:35:44.485294 10651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:44 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253744, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:44.491318 10651 init.cc:377] Logged in from keytab as oryx/127.30.137.193@KRBTEST.COM (short username oryx)
I20260809 05:35:44.492894 10651 webserver.cc:460] Webserver started at http://127.30.137.193:33223/ using document root <none> and password file <none>
I20260809 05:35:44.493646 10651 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:44.493777 10651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:44.494115 10651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:44.497169 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "b08e57effb7d4088911944f477594133"
format_stamp: "Formatted at 2026-08-09 05:35:44 on dist-test-slave-3kk6"
server_key: "752baca9926d69eee74bb731d5fb4bf4"
server_key_iv: "7b1ec7db71e75a8c3467cad90a3be287"
server_key_version: "encryptionkey@0"
I20260809 05:35:44.498041 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "b08e57effb7d4088911944f477594133"
format_stamp: "Formatted at 2026-08-09 05:35:44 on dist-test-slave-3kk6"
server_key: "752baca9926d69eee74bb731d5fb4bf4"
server_key_iv: "7b1ec7db71e75a8c3467cad90a3be287"
server_key_version: "encryptionkey@0"
I20260809 05:35:44.503435 10651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 05:35:44.507121 10672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:44.508420 10651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:35:44.508605 10651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "b08e57effb7d4088911944f477594133"
format_stamp: "Formatted at 2026-08-09 05:35:44 on dist-test-slave-3kk6"
server_key: "752baca9926d69eee74bb731d5fb4bf4"
server_key_iv: "7b1ec7db71e75a8c3467cad90a3be287"
server_key_version: "encryptionkey@0"
I20260809 05:35:44.508848 10651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:44.537185 10651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:44.541612 10651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:44.541934 10651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:44.544836 10651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:44.544953 10651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:44.545109 10651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:44.545189 10651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:35:44 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253744, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:44.571410 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:44.545622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:54715 (local address 127.30.137.254:44221)
0809 05:35:44.546244 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:44.546253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:44.547503 (+ 1250us) server_negotiation.cc:407] Connection header received
0809 05:35:44.549653 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:44.549659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:44.549754 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:44.549875 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:44.552963 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.554357 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.556740 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.557556 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.557713 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:44.563261 (+ 5548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:44.563266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:44.563270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:44.563307 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:44.566170 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:44.567079 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:44.567084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:44.567087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:44.567206 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:44.567832 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:44.567837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:44.567840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:44.568239 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:44.568404 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:44.570908 (+ 2504us) server_negotiation.cc:299] Negotiation successful
0809 05:35:44.571153 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":351,"threads_started":1}
I20260809 05:35:44.575448 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:44.546319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.30.137.193:54715)
0809 05:35:44.547289 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:44.547350 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:44.549352 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:44.550346 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:44.550359 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:44.551466 (+ 1107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:44.552626 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:44.552659 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.554580 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.554588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:44.556523 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:44.556548 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.557880 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.557888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:44.558098 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:44.559144 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:44.559185 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:44.562954 (+ 3769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:44.566333 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:44.566343 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:44.566360 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:44.566924 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:44.567331 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:44.567336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:44.567340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:44.567656 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:44.568488 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:44.568496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:44.569068 (+ 572us) client_negotiation.cc:769] Sending connection context
0809 05:35:44.569405 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 05:35:44.570063 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":406,"threads_started":1}
I20260809 05:35:44.596347 10651 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:46281
I20260809 05:35:44.596374 10856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:46281 every 8 connection(s)
I20260809 05:35:44.597942 10651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:44.600461 10858 heartbeater.cc:347] Connected to a master server at 127.30.137.254:44221
I20260809 05:35:44.600739 10858 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:44.601567 10858 heartbeater.cc:511] Master 127.30.137.254:44221 requested a full tablet report, sending...
I20260809 05:35:44.604085 10548 ts_manager.cc:194] Registered new tserver with Master: b08e57effb7d4088911944f477594133 (127.30.137.193:46281)
I20260809 05:35:44.604213 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:44.584306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:56795 (local address 127.30.137.254:44221)
0809 05:35:44.584534 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:44.584542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:44.587777 (+ 3235us) server_negotiation.cc:407] Connection header received
0809 05:35:44.588027 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:44.588037 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:44.588138 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:44.588272 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:44.589231 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.590520 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.593492 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.594198 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.594306 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:44.597170 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:44.597175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:44.597178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:44.597212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:44.601201 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:44.601959 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:44.601965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:44.601968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:44.602126 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:44.602527 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:44.602531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:44.602533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:44.602963 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:44.603159 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:44.603551 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 05:35:44.603778 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 05:35:44.604233 10821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:44.584335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.30.137.193:56795)
0809 05:35:44.587612 (+ 3277us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:44.587641 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:44.587848 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:44.588309 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:44.588312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:44.588762 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:44.589073 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:44.589083 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.590762 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.590771 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:44.593300 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:44.593324 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:44.594419 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:44.594429 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:44.594625 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:44.595617 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:44.595646 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:44.596979 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:44.601328 (+ 4349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:44.601332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:44.601335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:44.601803 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:44.602251 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:44.602254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:44.602257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:44.602402 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:44.603110 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:44.603114 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:44.603273 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 05:35:44.603422 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 05:35:44.603765 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3121,"thread_start_us":2972,"threads_started":1}
I20260809 05:35:44.606052 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 10651
I20260809 05:35:44.606259 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:44.606186 10548 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:54715
I20260809 05:35:44.607043 31271 external_mini_cluster.cc:1428] Setting key 5f018683b84743c4cd619d1bffd161de
WARNING: no policy specified for oryx/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab.
Entry for principal oryx/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab.
I20260809 05:35:44.666519 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44221
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:45.016896 10866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:45.017194 10866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:45.017351 10866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:45.024721 10866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:45.024873 10866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:45.025041 10866 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:35:45.025166 10866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:45.035100 10866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:44221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:45.036422 10866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:45.037765 10866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:45.051008 10878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:45.051050 10876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:45.050961 10875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:45.052018 10866 server_base.cc:1034] running on GCE node
I20260809 05:35:45.052707 10866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:45.053736 10866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:45.054984 10866 hybrid_clock.cc:648] HybridClock initialized: now 1786253745054914 us; error 44 us; skew 500 ppm
I20260809 05:35:45.055297 10866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253745, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:45.058743 10866 init.cc:377] Logged in from keytab as oryx/127.30.137.194@KRBTEST.COM (short username oryx)
I20260809 05:35:45.060346 10866 webserver.cc:460] Webserver started at http://127.30.137.194:36045/ using document root <none> and password file <none>
I20260809 05:35:45.061019 10866 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:45.061108 10866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:45.061399 10866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:45.064659 10866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
format_stamp: "Formatted at 2026-08-09 05:35:45 on dist-test-slave-3kk6"
server_key: "6095a74c0cbb34c4b2f28332cb1a0db0"
server_key_iv: "cde879749210d0fec1e3d3cde57c3af7"
server_key_version: "encryptionkey@0"
I20260809 05:35:45.065578 10866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
format_stamp: "Formatted at 2026-08-09 05:35:45 on dist-test-slave-3kk6"
server_key: "6095a74c0cbb34c4b2f28332cb1a0db0"
server_key_iv: "cde879749210d0fec1e3d3cde57c3af7"
server_key_version: "encryptionkey@0"
I20260809 05:35:45.070835 10866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:35:45.074213 10890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:45.075621 10866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:45.075785 10866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
format_stamp: "Formatted at 2026-08-09 05:35:45 on dist-test-slave-3kk6"
server_key: "6095a74c0cbb34c4b2f28332cb1a0db0"
server_key_iv: "cde879749210d0fec1e3d3cde57c3af7"
server_key_version: "encryptionkey@0"
I20260809 05:35:45.076032 10866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:45.121464 10866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:45.125633 10866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:45.125954 10866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:45.128793 10866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:45.128916 10866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:45.129154 10866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:45.129279 10866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253745, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:45.153039 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.129274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:59929 (local address 127.30.137.254:44221)
0809 05:35:45.129760 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.129766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.131082 (+ 1316us) server_negotiation.cc:407] Connection header received
0809 05:35:45.133207 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.133213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.133311 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.133432 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:45.135777 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.136947 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.138881 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.139542 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.139683 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.144913 (+ 5230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:45.144921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:45.144926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:45.144978 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:45.148931 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.149848 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.149857 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.149860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.150036 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.150633 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.150639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.150642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.150967 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:45.151176 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.152362 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.152624 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":242,"threads_started":1}
I20260809 05:35:45.154155 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.129957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.30.137.194:59929)
0809 05:35:45.130892 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.130943 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.132911 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.133686 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.133697 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.134468 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:45.135560 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.135596 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.137123 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.137128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.138644 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.138681 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.139817 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.139826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.140153 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.141106 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:45.141154 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:45.144645 (+ 3491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:45.149092 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.149101 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.149118 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.149678 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.150192 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.150197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.150200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.150490 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.151280 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:45.151290 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:45.151801 (+ 511us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.152236 (+ 435us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.152835 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":392,"threads_started":1}
I20260809 05:35:45.178570 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.160171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.30.137.194:60007)
0809 05:35:45.160886 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.160909 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.161028 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.161400 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.161403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.161877 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:45.162198 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.162207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.164029 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.164037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.166309 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.166329 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.167893 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.167902 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.168211 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.168952 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:45.168983 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:45.170403 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:45.174490 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.174499 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.174503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.175319 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.175839 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.175844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.175851 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.176048 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.177068 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:45.177073 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:45.177377 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.177585 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.178016 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":394,"threads_started":1}
I20260809 05:35:45.179803 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.160201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:60007 (local address 127.30.137.254:44221)
0809 05:35:45.160408 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.160415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.160994 (+ 579us) server_negotiation.cc:407] Connection header received
0809 05:35:45.161130 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.161138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.161239 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.161379 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:45.162345 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.163805 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.166478 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.167389 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.167551 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.170699 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:45.170709 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:45.170715 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:45.170759 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:45.174315 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.175551 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.175557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.175560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.175726 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.176196 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.176204 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.176208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.176784 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:45.176970 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.179018 (+ 2048us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.179337 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:35:45.188699 10866 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:42233
I20260809 05:35:45.188707 11063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:42233 every 8 connection(s)
I20260809 05:35:45.190070 10866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:45.192497 11064 heartbeater.cc:347] Connected to a master server at 127.30.137.254:44221
I20260809 05:35:45.192716 11064 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:45.193444 11064 heartbeater.cc:511] Master 127.30.137.254:44221 requested a full tablet report, sending...
I20260809 05:35:45.194861 10548 ts_manager.cc:194] Registered new tserver with Master: 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:45.195667 10548 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:59929
I20260809 05:35:45.198537 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 10866
I20260809 05:35:45.198755 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:45.199235 31271 external_mini_cluster.cc:1428] Setting key 4abf8d6626911eee98d8a918e130279a
WARNING: no policy specified for oryx/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab.
Entry for principal oryx/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab.
I20260809 05:35:45.259967 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44221
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:35:45.609606 10858 heartbeater.cc:503] Master 127.30.137.254:44221 was elected leader, sending a full tablet report...
W20260809 05:35:45.613427 11070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:45.613758 11070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:45.613926 11070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:45.621270 11070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:45.621434 11070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:45.621577 11070 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 05:35:45.621709 11070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:45.631763 11070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.137.254:44221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:45.633062 11070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:45.634408 11070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:45.648002 11079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:45.647934 11084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:45.647977 11080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:45.649351 11070 server_base.cc:1034] running on GCE node
I20260809 05:35:45.650007 11070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:45.651067 11070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:45.652326 11070 hybrid_clock.cc:648] HybridClock initialized: now 1786253745652232 us; error 75 us; skew 500 ppm
I20260809 05:35:45.652652 11070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253745, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:45.656149 11070 init.cc:377] Logged in from keytab as oryx/127.30.137.195@KRBTEST.COM (short username oryx)
I20260809 05:35:45.657693 11070 webserver.cc:460] Webserver started at http://127.30.137.195:42425/ using document root <none> and password file <none>
I20260809 05:35:45.658429 11070 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:45.658520 11070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:45.658880 11070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:45.661908 11070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "d41bfb18c8754247bc6d973ec25b42cd"
format_stamp: "Formatted at 2026-08-09 05:35:45 on dist-test-slave-3kk6"
server_key: "b23ab02f1e226f51db8d39ae32e814b3"
server_key_iv: "af98e19125e3803962f0c40dc1255ff3"
server_key_version: "encryptionkey@0"
I20260809 05:35:45.662776 11070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "d41bfb18c8754247bc6d973ec25b42cd"
format_stamp: "Formatted at 2026-08-09 05:35:45 on dist-test-slave-3kk6"
server_key: "b23ab02f1e226f51db8d39ae32e814b3"
server_key_iv: "af98e19125e3803962f0c40dc1255ff3"
server_key_version: "encryptionkey@0"
I20260809 05:35:45.668133 11070 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 05:35:45.672305 11092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:45.673693 11070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:35:45.674119 11070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "d41bfb18c8754247bc6d973ec25b42cd"
format_stamp: "Formatted at 2026-08-09 05:35:45 on dist-test-slave-3kk6"
server_key: "b23ab02f1e226f51db8d39ae32e814b3"
server_key_iv: "af98e19125e3803962f0c40dc1255ff3"
server_key_version: "encryptionkey@0"
I20260809 05:35:45.674417 11070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:45.689808 11070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:45.695273 11070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:45.695585 11070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:45.698486 11070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:45.698606 11070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:45.698771 11070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:45.698853 11070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253745, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:45.736871 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.699242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:48241 (local address 127.30.137.254:44221)
0809 05:35:45.699782 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.699791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.701270 (+ 1479us) server_negotiation.cc:407] Connection header received
0809 05:35:45.703940 (+ 2670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.703947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.704053 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.704207 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:45.707650 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.709220 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.724459 (+ 15239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.724970 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.725144 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.729562 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:45.729568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:45.729572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:45.729611 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:45.732971 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.733868 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.733873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.733875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.734023 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.734532 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.734539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.734548 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.735024 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:45.735319 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.736135 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.736410 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":291,"threads_started":1}
I20260809 05:35:45.738149 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.700066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.30.137.195:48241)
0809 05:35:45.701053 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.701103 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.703536 (+ 2433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.704739 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.704757 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.705865 (+ 1108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:45.707339 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.707389 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.717620 (+ 10231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.717632 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.724195 (+ 6563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.724221 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.725295 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.725305 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.725548 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.726391 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:45.726419 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:45.729261 (+ 2842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:45.733155 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.733166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.733186 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.733701 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.734157 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.734164 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.734169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.734388 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.735271 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:45.735283 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:45.735714 (+ 431us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.736030 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.736673 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":465,"threads_started":1}
I20260809 05:35:45.760114 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.744668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:37211 (local address 127.30.137.254:44221)
0809 05:35:45.744878 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.744888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.745683 (+ 795us) server_negotiation.cc:407] Connection header received
0809 05:35:45.745907 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.745913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.746011 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.746130 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:45.747464 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.748280 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.749890 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.750380 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.750528 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.752905 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:45.752918 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:45.752922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:45.752958 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:45.756352 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.757304 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.757312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.757316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.757506 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.757989 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.757995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.757999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.758378 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:45.758558 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.759556 (+ 998us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.759831 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:35:45.761149 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.744699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.30.137.195:37211)
0809 05:35:45.745546 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.745567 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.745760 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.746169 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.746176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.746825 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:45.747302 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.747315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.748450 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.748456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.749725 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.749739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.750608 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.750615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.750835 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.751640 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:45.751675 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:45.752694 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:45.756502 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.756508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.756513 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.757126 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.757631 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:45.757637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:45.757641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:45.757858 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:45.758565 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:45.758572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:45.759072 (+ 500us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.760450 (+ 1378us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.760787 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":503,"threads_started":1}
I20260809 05:35:45.793021 11070 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:41513
I20260809 05:35:45.793082 11281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:41513 every 8 connection(s)
I20260809 05:35:45.795168 11070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:45.797932 11282 heartbeater.cc:347] Connected to a master server at 127.30.137.254:44221
I20260809 05:35:45.798216 11282 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:45.799041 11282 heartbeater.cc:511] Master 127.30.137.254:44221 requested a full tablet report, sending...
I20260809 05:35:45.800037 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 11070
I20260809 05:35:45.800252 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:45.800858 31271 external_mini_cluster.cc:1428] Setting key 98109a053408457bf1a7138418c23e99
I20260809 05:35:45.800836 10548 ts_manager.cc:194] Registered new tserver with Master: d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513)
I20260809 05:35:45.801928 10548 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:48241
I20260809 05:35:45.804572 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 05:35:45.885277 11104 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:45.870462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.30.137.254:44221)
0809 05:35:45.870779 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.870786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.871247 (+ 461us) server_negotiation.cc:407] Connection header received
0809 05:35:45.871492 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.871501 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.871586 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.871691 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:45.873816 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.874573 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.876939 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.877390 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.877498 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.884915 (+ 7417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53446: BlockingRecv error: recv got EOF from 127.0.0.1:53446 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 05:35:45 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253745, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:45.908835 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.889062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.30.137.254:44221)
0809 05:35:45.889279 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.889288 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.889638 (+ 350us) server_negotiation.cc:407] Connection header received
0809 05:35:45.889821 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.889830 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.889955 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.890099 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:45.891356 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.892746 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.894172 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.895085 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.895256 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.899710 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:45.899715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:45.899719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:45.899760 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:45.904273 (+ 4513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.905471 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.905479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.905483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.905689 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:45.906236 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:45.906244 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:45.906248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:45.906702 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:45.907017 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.908239 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.908503 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:35:45.919898 10548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:45.926913 10548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:45.980152 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.948504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.30.137.195:41513)
0809 05:35:45.964542 (+ 16038us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.964555 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.966893 (+ 2338us) server_negotiation.cc:407] Connection header received
0809 05:35:45.967189 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.967197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.967504 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.967731 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:45.971501 (+ 3770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.973415 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.976463 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.978134 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.978308 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.978616 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.979362 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.979628 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15777,"thread_start_us":16799,"threads_started":1}
I20260809 05:35:45.981336 11316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.940093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:42233 (local address 127.0.0.1:37098)
0809 05:35:45.963241 (+ 23148us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.963313 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.963620 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.971671 (+ 8051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.971679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.971716 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:45.972191 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.972208 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.974803 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.974811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.977718 (+ 2907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.977737 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.980179 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.980189 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.980425 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.980474 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.980549 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.980808 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22720,"mutex_wait_us":106,"thread_start_us":24487,"threads_started":1}
I20260809 05:35:45.982004 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.940772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.30.137.194:42233)
0809 05:35:45.970226 (+ 29454us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.970236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.970268 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:35:45.970365 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.970382 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.970642 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.972472 (+ 1830us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:45.972769 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.974545 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.977951 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.979909 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.981090 (+ 1181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.981305 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.981392 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.981596 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29221,"thread_start_us":30096,"threads_started":1}
I20260809 05:35:45.982438 11322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.948162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:41513 (local address 127.0.0.1:36266)
0809 05:35:45.966775 (+ 18613us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.966794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.966989 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.968235 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.968248 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.968289 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:45.971224 (+ 2935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.971241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.973614 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.973622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.976236 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.976252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.978876 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.978883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.979075 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.979130 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.980690 (+ 1560us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.980842 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18459,"mutex_wait_us":1,"thread_start_us":18327,"threads_started":1}
I20260809 05:35:45.983309 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.940579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39058 (local address 127.30.137.193:46281)
0809 05:35:45.971576 (+ 30997us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:45.971586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:45.971617 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:35:45.971737 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:45.971744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:45.971984 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:45.972890 (+ 906us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:45.973118 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.975939 (+ 2821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.980129 (+ 4190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.981800 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.982606 (+ 806us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.982792 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:45.982855 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 05:35:45.983020 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30825,"thread_start_us":30676,"threads_started":1}
I20260809 05:35:45.984182 11318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:45.940199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:46281 (local address 127.0.0.1:39058)
0809 05:35:45.965979 (+ 25780us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:45.965997 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:45.966127 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:45.972233 (+ 6106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:45.972243 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:45.972271 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:45.972658 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.972670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.976126 (+ 3456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.976132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.979908 (+ 3776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:45.979923 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:45.982071 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:45.982079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:45.982249 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:45.982289 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 05:35:45.982360 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 05:35:45.982530 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25659,"mutex_wait_us":33,"thread_start_us":25506,"threads_started":1}
I20260809 05:35:45.986609 11190 tablet_service.cc:1467] Processing CreateTablet for tablet 79bd38dffa714dd5b5237d6c0474abdd (DEFAULT_TABLE table=test-table [id=b545ae965fe84d9d94111e54c41a087c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:45.986619 10954 tablet_service.cc:1467] Processing CreateTablet for tablet 79bd38dffa714dd5b5237d6c0474abdd (DEFAULT_TABLE table=test-table [id=b545ae965fe84d9d94111e54c41a087c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:45.988256 10954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd38dffa714dd5b5237d6c0474abdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:45.988365 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd38dffa714dd5b5237d6c0474abdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:45.988039 10752 tablet_service.cc:1467] Processing CreateTablet for tablet 79bd38dffa714dd5b5237d6c0474abdd (DEFAULT_TABLE table=test-table [id=b545ae965fe84d9d94111e54c41a087c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:45.989450 10752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd38dffa714dd5b5237d6c0474abdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.009235 11327 tablet_bootstrap.cc:492] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Bootstrap starting.
I20260809 05:35:46.015307 11330 tablet_bootstrap.cc:492] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Bootstrap starting.
I20260809 05:35:46.016016 11327 tablet_bootstrap.cc:654] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.017606 11327 log.cc:824] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:46.018528 11325 tablet_bootstrap.cc:492] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Bootstrap starting.
I20260809 05:35:46.022773 11325 tablet_bootstrap.cc:654] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.023530 11330 tablet_bootstrap.cc:654] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.024273 11325 log.cc:824] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:46.025175 11330 log.cc:824] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:46.030459 11327 tablet_bootstrap.cc:492] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: No bootstrap required, opened a new log
I20260809 05:35:46.030452 11330 tablet_bootstrap.cc:492] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: No bootstrap required, opened a new log
I20260809 05:35:46.030779 11330 ts_tablet_manager.cc:1397] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.003s
I20260809 05:35:46.030802 11327 ts_tablet_manager.cc:1397] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.005s
I20260809 05:35:46.033715 11325 tablet_bootstrap.cc:492] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: No bootstrap required, opened a new log
I20260809 05:35:46.034096 11325 ts_tablet_manager.cc:1397] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260809 05:35:46.043154 11330 raft_consensus.cc:348] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.043766 11330 raft_consensus.cc:374] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.043910 11330 raft_consensus.cc:729] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.044916 11330 consensus_queue.cc:260] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.045490 11327 raft_consensus.cc:348] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.046134 11327 raft_consensus.cc:374] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.046332 11327 raft_consensus.cc:729] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285fecbb8f0c4d30b5e1f12d1f743b53, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.047734 11327 consensus_queue.cc:260] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.048399 11325 raft_consensus.cc:348] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.049010 11325 raft_consensus.cc:374] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.049206 11325 raft_consensus.cc:729] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b08e57effb7d4088911944f477594133, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.050068 11325 consensus_queue.cc:260] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.052456 11282 heartbeater.cc:503] Master 127.30.137.254:44221 was elected leader, sending a full tablet report...
I20260809 05:35:46.053627 11330 ts_tablet_manager.cc:1428] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Time spent starting tablet: real 0.023s user 0.012s sys 0.007s
I20260809 05:35:46.054759 11064 heartbeater.cc:503] Master 127.30.137.254:44221 was elected leader, sending a full tablet report...
I20260809 05:35:46.058436 11327 ts_tablet_manager.cc:1428] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent starting tablet: real 0.027s user 0.012s sys 0.007s
I20260809 05:35:46.058480 11325 ts_tablet_manager.cc:1428] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Time spent starting tablet: real 0.024s user 0.015s sys 0.006s
W20260809 05:35:46.102445 10860 tablet.cc:2369] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:46.102890 11338 raft_consensus.cc:482] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:46.103227 11338 raft_consensus.cc:504] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.105491 11338 leader_election.cc:288] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513)
I20260809 05:35:46.115142 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.106381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:50665 (local address 127.30.137.193:46281)
0809 05:35:46.106787 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.106796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.107367 (+ 571us) server_negotiation.cc:407] Connection header received
0809 05:35:46.107609 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.107613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.107685 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.107767 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:46.108392 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.109502 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.112533 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.113827 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.113981 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.114045 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.114748 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.114893 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260809 05:35:46.115150 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.106785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:33117 (local address 127.30.137.195:41513)
0809 05:35:46.107040 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.107049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.107688 (+ 639us) server_negotiation.cc:407] Connection header received
0809 05:35:46.107896 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.107901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.107976 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.108059 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:46.108715 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.109737 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.112502 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.113785 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.113969 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.114045 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.114748 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.114881 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:35:46.115412 11342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.106484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:41513 (local address 127.30.137.194:33117)
0809 05:35:46.107578 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.107596 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.107731 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.108110 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.108118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.108150 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:46.108544 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.108553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.109865 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.109872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.112316 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.112333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.114094 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.114106 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.114462 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.114579 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.114706 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.114909 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":287,"thread_start_us":744,"threads_started":1}
I20260809 05:35:46.115448 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.105829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:46281 (local address 127.30.137.194:50665)
0809 05:35:46.107227 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.107253 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.107449 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.107792 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.107799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.107828 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:46.108226 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.108243 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.109644 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.109648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.112315 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.112333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.114118 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.114123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.114462 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.114578 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.114706 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.114909 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1217,"mutex_wait_us":175,"thread_start_us":1034,"threads_started":1}
I20260809 05:35:46.116452 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd38dffa714dd5b5237d6c0474abdd" candidate_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133" is_pre_election: true
I20260809 05:35:46.116493 11219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd38dffa714dd5b5237d6c0474abdd" candidate_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d41bfb18c8754247bc6d973ec25b42cd" is_pre_election: true
I20260809 05:35:46.116989 11219 raft_consensus.cc:2454] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285fecbb8f0c4d30b5e1f12d1f743b53 in term 0.
I20260809 05:35:46.116989 10784 raft_consensus.cc:2454] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 285fecbb8f0c4d30b5e1f12d1f743b53 in term 0.
I20260809 05:35:46.117655 10895 leader_election.cc:302] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285fecbb8f0c4d30b5e1f12d1f743b53, b08e57effb7d4088911944f477594133; no voters:
I20260809 05:35:46.118078 11338 raft_consensus.cc:2781] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:46.118229 11338 raft_consensus.cc:482] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:46.118335 11338 raft_consensus.cc:3037] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.120656 11338 raft_consensus.cc:504] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.121291 11338 leader_election.cc:288] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [CANDIDATE]: Term 1 election: Requested vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513)
I20260809 05:35:46.121757 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd38dffa714dd5b5237d6c0474abdd" candidate_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133"
I20260809 05:35:46.121855 11219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd38dffa714dd5b5237d6c0474abdd" candidate_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d41bfb18c8754247bc6d973ec25b42cd"
I20260809 05:35:46.121989 10784 raft_consensus.cc:3037] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.122066 11219 raft_consensus.cc:3037] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.125339 11219 raft_consensus.cc:2454] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285fecbb8f0c4d30b5e1f12d1f743b53 in term 1.
I20260809 05:35:46.125339 10784 raft_consensus.cc:2454] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 285fecbb8f0c4d30b5e1f12d1f743b53 in term 1.
I20260809 05:35:46.125801 10894 leader_election.cc:302] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285fecbb8f0c4d30b5e1f12d1f743b53, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:46.126163 11338 raft_consensus.cc:2781] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:46.126688 11338 raft_consensus.cc:686] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 LEADER]: Becoming Leader. State: Replica: 285fecbb8f0c4d30b5e1f12d1f743b53, State: Running, Role: LEADER
I20260809 05:35:46.127221 11338 consensus_queue.cc:237] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.131927 10547 catalog_manager.cc:5495] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194). New cstate: current_term: 1 leader_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } health_report { overall_health: UNKNOWN } } }
I20260809 05:35:46.182080 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.176719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.30.137.194:42233)
0809 05:35:46.176927 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.176932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.176950 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:35:46.177040 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.177044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.177130 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.177252 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:46.177574 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.178415 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.179977 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.180448 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.180546 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.180970 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:46.181506 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:46.181614 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.181690 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.181756 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:35:46.190477 10784 raft_consensus.cc:1261] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Refusing update from remote peer 285fecbb8f0c4d30b5e1f12d1f743b53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:46.190584 11219 raft_consensus.cc:1261] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Refusing update from remote peer 285fecbb8f0c4d30b5e1f12d1f743b53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:46.191592 11345 consensus_queue.cc:1035] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:46.192013 11338 consensus_queue.cc:1035] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:35:46.192384 11066 tablet.cc:2369] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:46.223402 11351 mvcc.cc:205] Tried to move back new op lower bound from 7316495344386240512 to 7316495344140869632. Current Snapshot: MvccSnapshot[applied={T|T < 7316495344386240512}]
I20260809 05:35:46.228408 10547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:46.228696 10547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:46.232206 10547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Sending DeleteTablet for 3 replicas of tablet 79bd38dffa714dd5b5237d6c0474abdd
I20260809 05:35:46.233316 10954 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bd38dffa714dd5b5237d6c0474abdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:46 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:37098
I20260809 05:35:46.233371 10752 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bd38dffa714dd5b5237d6c0474abdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:46 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:39058
I20260809 05:35:46.233459 11190 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bd38dffa714dd5b5237d6c0474abdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:46 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36266
I20260809 05:35:46.236358 10547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:46.237910 10547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:46.243819 11368 tablet_replica.cc:330] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: stopping tablet replica
I20260809 05:35:46.244410 11368 raft_consensus.cc:2229] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:46.245011 11366 tablet_replica.cc:330] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: stopping tablet replica
I20260809 05:35:46.245543 11368 raft_consensus.cc:2258] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:46.246021 11366 raft_consensus.cc:2229] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:46.246793 11366 raft_consensus.cc:2258] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:46.248947 11366 ts_tablet_manager.cc:1905] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:46.251925 10953 tablet_service.cc:1467] Processing CreateTablet for tablet a9c47f22f65c40c9b7fc73353e294705 (DEFAULT_TABLE table=test-table [id=43866376e8994397a9fd17f9adc959c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:46.252880 11190 tablet_service.cc:1467] Processing CreateTablet for tablet a9c47f22f65c40c9b7fc73353e294705 (DEFAULT_TABLE table=test-table [id=43866376e8994397a9fd17f9adc959c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:46.253424 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c47f22f65c40c9b7fc73353e294705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.253834 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c47f22f65c40c9b7fc73353e294705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.254714 10752 tablet_service.cc:1467] Processing CreateTablet for tablet a9c47f22f65c40c9b7fc73353e294705 (DEFAULT_TABLE table=test-table [id=43866376e8994397a9fd17f9adc959c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:46.255124 10752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c47f22f65c40c9b7fc73353e294705. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.256070 11368 ts_tablet_manager.cc:1905] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:46.260694 11330 tablet_bootstrap.cc:492] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Bootstrap starting.
I20260809 05:35:46.261138 11365 tablet_replica.cc:330] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: stopping tablet replica
I20260809 05:35:46.261646 11325 tablet_bootstrap.cc:492] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Bootstrap starting.
I20260809 05:35:46.261646 11365 raft_consensus.cc:2229] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:46.262006 11327 tablet_bootstrap.cc:492] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Bootstrap starting.
I20260809 05:35:46.262456 11330 tablet_bootstrap.cc:654] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.262598 11365 raft_consensus.cc:2258] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:46.263286 11325 tablet_bootstrap.cc:654] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.263726 11327 tablet_bootstrap.cc:654] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.270210 11365 ts_tablet_manager.cc:1905] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:46.274122 11366 ts_tablet_manager.cc:1918] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:46.275725 11325 tablet_bootstrap.cc:492] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: No bootstrap required, opened a new log
I20260809 05:35:46.275885 11325 ts_tablet_manager.cc:1397] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260809 05:35:46.276422 11330 tablet_bootstrap.cc:492] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: No bootstrap required, opened a new log
I20260809 05:35:46.276458 11327 tablet_bootstrap.cc:492] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: No bootstrap required, opened a new log
I20260809 05:35:46.276588 11330 ts_tablet_manager.cc:1397] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.006s
I20260809 05:35:46.276623 11327 ts_tablet_manager.cc:1397] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260809 05:35:46.277401 11330 raft_consensus.cc:348] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.277536 11327 raft_consensus.cc:348] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.277670 11330 raft_consensus.cc:374] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.277781 11327 raft_consensus.cc:374] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.277788 11330 raft_consensus.cc:729] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.277920 11327 raft_consensus.cc:729] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285fecbb8f0c4d30b5e1f12d1f743b53, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.276799 11325 raft_consensus.cc:348] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.278196 11325 raft_consensus.cc:374] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.278154 11330 consensus_queue.cc:260] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.278309 11325 raft_consensus.cc:729] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b08e57effb7d4088911944f477594133, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.278568 11327 consensus_queue.cc:260] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.278625 11325 consensus_queue.cc:260] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.280380 11327 ts_tablet_manager.cc:1428] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 05:35:46.281534 11325 ts_tablet_manager.cc:1428] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 05:35:46.275695 11366 log.cc:1196] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/79bd38dffa714dd5b5237d6c0474abdd
I20260809 05:35:46.282485 11330 ts_tablet_manager.cc:1428] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260809 05:35:46.283480 11366 ts_tablet_manager.cc:1939] T 79bd38dffa714dd5b5237d6c0474abdd P b08e57effb7d4088911944f477594133: Deleting consensus metadata
I20260809 05:35:46.285971 11368 ts_tablet_manager.cc:1918] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:46.286154 11368 log.cc:1196] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/79bd38dffa714dd5b5237d6c0474abdd
I20260809 05:35:46.286739 11368 ts_tablet_manager.cc:1939] T 79bd38dffa714dd5b5237d6c0474abdd P d41bfb18c8754247bc6d973ec25b42cd: Deleting consensus metadata
I20260809 05:35:46.288391 10526 catalog_manager.cc:4841] TS d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513): tablet 79bd38dffa714dd5b5237d6c0474abdd (table test-table [id=b545ae965fe84d9d94111e54c41a087c]) successfully deleted
I20260809 05:35:46.289280 10530 catalog_manager.cc:4841] TS b08e57effb7d4088911944f477594133 (127.30.137.193:46281): tablet 79bd38dffa714dd5b5237d6c0474abdd (table test-table [id=b545ae965fe84d9d94111e54c41a087c]) successfully deleted
I20260809 05:35:46.290570 11365 ts_tablet_manager.cc:1918] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:46.290809 11365 log.cc:1196] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/79bd38dffa714dd5b5237d6c0474abdd
I20260809 05:35:46.291811 11365 ts_tablet_manager.cc:1939] T 79bd38dffa714dd5b5237d6c0474abdd P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting consensus metadata
I20260809 05:35:46.293546 10525 catalog_manager.cc:4841] TS 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233): tablet 79bd38dffa714dd5b5237d6c0474abdd (table test-table [id=b545ae965fe84d9d94111e54c41a087c]) successfully deleted
W20260809 05:35:46.297307 11283 tablet.cc:2369] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:35:46.329053 11337 raft_consensus.cc:482] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:46.329339 11337 raft_consensus.cc:504] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.331010 11337 leader_election.cc:288] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:46.338580 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.331535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:33723 (local address 127.30.137.193:46281)
0809 05:35:46.331937 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.331944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.331967 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:35:46.332255 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.332261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.332347 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.332425 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:46.333258 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.334421 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.336837 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.337645 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.337827 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.337913 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.338178 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.338308 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251}
I20260809 05:35:46.338929 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.331226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:46281 (local address 127.30.137.195:33723)
0809 05:35:46.331862 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.331890 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.332049 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.332650 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.332653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.332689 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:46.333045 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.333061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.334556 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.334561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.336678 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.336691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.337828 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.337834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.338021 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.338045 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.338152 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.338279 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":49,"spinlock_wait_cycles":2432,"thread_start_us":389,"threads_started":1}
I20260809 05:35:46.339576 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.332117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:60259 (local address 127.30.137.194:42233)
0809 05:35:46.332308 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.332313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.332331 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:35:46.332412 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.332417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.332491 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.332586 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:46.333393 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.334144 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.336749 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.338059 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.338217 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.338303 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.339125 (+ 822us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.339264 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:35:46.339833 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c47f22f65c40c9b7fc73353e294705" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133" is_pre_election: true
I20260809 05:35:46.340102 10784 raft_consensus.cc:2454] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 0.
I20260809 05:35:46.340345 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.331400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:42233 (local address 127.30.137.195:60259)
0809 05:35:46.332182 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.332200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.332286 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.332639 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.332644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.332682 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:46.333191 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.333202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.334285 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.334290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.336542 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.336558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.338469 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.338476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.338924 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.338960 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.339823 (+ 863us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.339979 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":101,"spinlock_wait_cycles":896,"thread_start_us":951,"threads_started":1}
I20260809 05:35:46.340678 11096 leader_election.cc:302] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:46.341118 11337 raft_consensus.cc:2781] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:46.341233 11337 raft_consensus.cc:482] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:46.341334 11337 raft_consensus.cc:3037] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.341257 10998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c47f22f65c40c9b7fc73353e294705" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" is_pre_election: true
I20260809 05:35:46.341741 10998 raft_consensus.cc:2454] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 0.
I20260809 05:35:46.344003 11337 raft_consensus.cc:504] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.344697 11337 leader_election.cc:288] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 election: Requested vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:46.345121 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c47f22f65c40c9b7fc73353e294705" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133"
I20260809 05:35:46.345224 10998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c47f22f65c40c9b7fc73353e294705" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
I20260809 05:35:46.345331 10784 raft_consensus.cc:3037] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.345456 10998 raft_consensus.cc:3037] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.347558 10784 raft_consensus.cc:2454] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 1.
I20260809 05:35:46.347985 11096 leader_election.cc:302] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:46.348278 11337 raft_consensus.cc:2781] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:46.348802 11337 raft_consensus.cc:686] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 1 LEADER]: Becoming Leader. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Running, Role: LEADER
I20260809 05:35:46.348958 10998 raft_consensus.cc:2454] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 1.
I20260809 05:35:46.349498 11337 consensus_queue.cc:237] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.353170 10548 catalog_manager.cc:5495] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195). New cstate: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } health_report { overall_health: HEALTHY } } }
Aug 09 05:35:46 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253743, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.254@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:46.413504 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.397528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44221 (local address 127.0.0.1:53468)
0809 05:35:46.398274 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.398291 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.398430 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.398826 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.398830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.399529 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:46.399828 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.399838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.401347 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.401350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.402287 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.402298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.403333 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.403338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.403462 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.404251 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:46.404269 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:46.406589 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:46.410740 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:46.410753 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:46.410759 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:46.411206 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:46.411604 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:46.411607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:46.411609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:46.411768 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:46.412496 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:46.412503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:46.412675 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.412818 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.413155 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":408,"threads_started":1}
I20260809 05:35:46.413789 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.397518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.30.137.254:44221)
0809 05:35:46.397754 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.397762 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.398397 (+ 635us) server_negotiation.cc:407] Connection header received
0809 05:35:46.398562 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.398568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.398673 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.398795 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:46.399992 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.401193 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.402450 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.403153 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.403293 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.406851 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:46.406859 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:46.406864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:46.406919 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:46.410507 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:46.411330 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:46.411336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:46.411340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:46.411488 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:46.411952 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:46.411958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:46.411986 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:46.412365 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:46.412524 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.413004 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.413364 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":81}
I20260809 05:35:46.418998 10548 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:53468:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 05:35:46.420056 10548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:46.427027 10752 tablet_service.cc:1467] Processing CreateTablet for tablet d9c70542333745e9bc44c2a58639dd1d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6b119aef53a84dc49631ca7d94cd8a44]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:35:46.427656 10752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c70542333745e9bc44c2a58639dd1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.427434 11190 tablet_service.cc:1467] Processing CreateTablet for tablet d9c70542333745e9bc44c2a58639dd1d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6b119aef53a84dc49631ca7d94cd8a44]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:35:46.428004 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c70542333745e9bc44c2a58639dd1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.431275 10953 tablet_service.cc:1467] Processing CreateTablet for tablet d9c70542333745e9bc44c2a58639dd1d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6b119aef53a84dc49631ca7d94cd8a44]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 05:35:46.431879 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c70542333745e9bc44c2a58639dd1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.435137 11325 tablet_bootstrap.cc:492] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: Bootstrap starting.
I20260809 05:35:46.435139 11330 tablet_bootstrap.cc:492] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: Bootstrap starting.
I20260809 05:35:46.436882 11330 tablet_bootstrap.cc:654] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.436887 11325 tablet_bootstrap.cc:654] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.440323 11327 tablet_bootstrap.cc:492] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: Bootstrap starting.
I20260809 05:35:46.442167 11327 tablet_bootstrap.cc:654] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.447824 11327 tablet_bootstrap.cc:492] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: No bootstrap required, opened a new log
I20260809 05:35:46.448026 11327 ts_tablet_manager.cc:1397] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 05:35:46.448935 11330 tablet_bootstrap.cc:492] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: No bootstrap required, opened a new log
I20260809 05:35:46.448860 11327 raft_consensus.cc:348] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.449144 11327 raft_consensus.cc:374] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.449235 11327 raft_consensus.cc:729] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285fecbb8f0c4d30b5e1f12d1f743b53, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.449395 11330 ts_tablet_manager.cc:1397] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260809 05:35:46.449483 11327 consensus_queue.cc:260] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.450109 11327 ts_tablet_manager.cc:1428] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 05:35:46.449975 11345 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.450351 11345 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.450627 11330 raft_consensus.cc:348] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.450970 11330 raft_consensus.cc:374] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.451100 11330 raft_consensus.cc:729] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.451154 11325 tablet_bootstrap.cc:492] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: No bootstrap required, opened a new log
I20260809 05:35:46.451335 11325 ts_tablet_manager.cc:1397] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260809 05:35:46.451493 11330 consensus_queue.cc:260] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.452162 11330 ts_tablet_manager.cc:1428] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 05:35:46.452028 11337 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.453235 11337 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.453728 11325 raft_consensus.cc:348] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.453986 11325 raft_consensus.cc:374] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.454052 11325 raft_consensus.cc:729] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b08e57effb7d4088911944f477594133, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.454275 11325 consensus_queue.cc:260] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.454913 11325 ts_tablet_manager.cc:1428] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 05:35:46.454823 11339 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.455241 11339 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.592382 11337 raft_consensus.cc:482] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:46.592674 11337 raft_consensus.cc:504] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.593425 11337 leader_election.cc:288] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:46.594138 10998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c70542333745e9bc44c2a58639dd1d" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" is_pre_election: true
I20260809 05:35:46.594177 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c70542333745e9bc44c2a58639dd1d" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133" is_pre_election: true
I20260809 05:35:46.594440 10998 raft_consensus.cc:2454] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 0.
I20260809 05:35:46.594523 10784 raft_consensus.cc:2454] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 0.
I20260809 05:35:46.595108 11094 leader_election.cc:302] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 285fecbb8f0c4d30b5e1f12d1f743b53, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:46.595538 11337 raft_consensus.cc:2781] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:46.595651 11337 raft_consensus.cc:482] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:46.595870 11337 raft_consensus.cc:3037] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.599156 11337 raft_consensus.cc:504] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.600090 11337 leader_election.cc:288] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 election: Requested vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:46.600436 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c70542333745e9bc44c2a58639dd1d" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133"
I20260809 05:35:46.600685 10784 raft_consensus.cc:3037] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.602800 10998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c70542333745e9bc44c2a58639dd1d" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
I20260809 05:35:46.603143 10998 raft_consensus.cc:3037] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:46.603883 10784 raft_consensus.cc:2454] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 1.
I20260809 05:35:46.604364 11096 leader_election.cc:302] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:46.604746 11337 raft_consensus.cc:2781] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:46.605224 11337 raft_consensus.cc:686] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [term 1 LEADER]: Becoming Leader. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Running, Role: LEADER
I20260809 05:35:46.605494 11337 consensus_queue.cc:237] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.607484 10998 raft_consensus.cc:2454] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 1.
I20260809 05:35:46.609040 11383 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: TxnStatusTablet state changed. Reason: New leader d41bfb18c8754247bc6d973ec25b42cd. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.609493 11383 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:35:46.610363 11403 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 05:35:46.610710 11403 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 05:35:46.610946 10548 catalog_manager.cc:5495] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195). New cstate: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } health_report { overall_health: HEALTHY } } }
I20260809 05:35:46.672541 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.664331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:41513 (local address 127.0.0.1:36274)
0809 05:35:46.664601 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.664623 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.664812 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.665762 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.665767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.665782 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:35:46.665986 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.665996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.667696 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.667702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.669278 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.669296 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.670312 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.670318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.670527 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.670602 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:35:46.671911 (+ 1309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:35:46.671941 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.672040 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.672215 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 05:35:46.672600 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.664474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.30.137.195:41513)
0809 05:35:46.665182 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.665188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.665209 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:35:46.665347 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.665352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.665483 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.665723 (+ 240us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:46.666188 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.667523 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.669449 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.670127 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.670265 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.670937 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:46.671764 (+ 827us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:46.671920 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.672082 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.672172 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":331,"threads_started":1}
I20260809 05:35:46.677886 10784 raft_consensus.cc:1261] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Refusing update from remote peer d41bfb18c8754247bc6d973ec25b42cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:46.678117 10998 raft_consensus.cc:1261] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Refusing update from remote peer d41bfb18c8754247bc6d973ec25b42cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:46.678822 11383 consensus_queue.cc:1035] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:46.680094 11383 consensus_queue.cc:1035] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:46.693396 11339 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: TxnStatusTablet state changed. Reason: New leader d41bfb18c8754247bc6d973ec25b42cd. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.693747 11339 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.694877 11345 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: TxnStatusTablet state changed. Reason: New leader d41bfb18c8754247bc6d973ec25b42cd. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.695199 11345 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.697284 11383 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.697556 11383 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:35:46.698201 11383 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.698429 11383 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P d41bfb18c8754247bc6d973ec25b42cd: This TxnStatusTablet replica's current role is: LEADER
I20260809 05:35:46.698629 11350 mvcc.cc:205] Tried to move back new op lower bound from 7316495346383384576 to 7316495346098597888. Current Snapshot: MvccSnapshot[applied={T|T < 7316495346383384576 or (T in {7316495346383384576})}]
I20260809 05:35:46.699836 11345 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.700122 11345 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P 285fecbb8f0c4d30b5e1f12d1f743b53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.703752 11339 tablet_replica.cc:439] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } } }
I20260809 05:35:46.704025 11339 tablet_replica.cc:442] T d9c70542333745e9bc44c2a58639dd1d P b08e57effb7d4088911944f477594133: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 05:35:46.718041 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.708837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.30.137.195:41513)
0809 05:35:46.709000 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.709006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.711402 (+ 2396us) server_negotiation.cc:407] Connection header received
0809 05:35:46.711728 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.711735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.711830 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.711942 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:46.712458 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.713769 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.715835 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.716528 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.716662 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.717166 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:46.717300 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:46.717408 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.717602 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.717692 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 05:35:46.737404 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.730775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:50131 (local address 127.30.137.195:41513)
0809 05:35:46.730973 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.730977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.731609 (+ 632us) server_negotiation.cc:407] Connection header received
0809 05:35:46.731793 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.731799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.731902 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.732000 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:46.732536 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.733721 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.735292 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.735998 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.736129 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.736576 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:46.736710 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:46.736805 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.736947 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.737037 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":62}
I20260809 05:35:46.737442 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.730847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:41513 (local address 127.30.137.195:50131)
0809 05:35:46.731503 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:46.731519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:46.731663 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:46.732033 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:46.732036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:46.732045 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:35:46.732398 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.732406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.733857 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.733860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.735154 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:46.735168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.736185 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.736190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:46.736331 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.736390 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:35:46.736821 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:35:46.736838 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 05:35:46.736900 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 05:35:46.737006 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":352,"threads_started":1}
I20260809 05:35:46.748376 10784 raft_consensus.cc:1261] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Refusing update from remote peer d41bfb18c8754247bc6d973ec25b42cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:46.748407 10998 raft_consensus.cc:1261] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Refusing update from remote peer d41bfb18c8754247bc6d973ec25b42cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:46.749058 11383 consensus_queue.cc:1035] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:46.749368 11337 consensus_queue.cc:1035] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:46.771672 11349 mvcc.cc:205] Tried to move back new op lower bound from 7316495346676674560 to 7316495345051377664. Current Snapshot: MvccSnapshot[applied={T|T < 7316495346676674560}]
I20260809 05:35:46.825250 10546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:46.825611 10546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:46.829267 10546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Sending DeleteTablet for 3 replicas of tablet a9c47f22f65c40c9b7fc73353e294705
I20260809 05:35:46.830308 10752 tablet_service.cc:1514] Processing DeleteTablet for tablet a9c47f22f65c40c9b7fc73353e294705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:46 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:39058
I20260809 05:35:46.830354 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet a9c47f22f65c40c9b7fc73353e294705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:46 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:37098
I20260809 05:35:46.839080 11426 tablet_replica.cc:330] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: stopping tablet replica
I20260809 05:35:46.839371 11426 raft_consensus.cc:2229] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:46.839604 11426 raft_consensus.cc:2258] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:46.840294 11427 tablet_replica.cc:330] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: stopping tablet replica
I20260809 05:35:46.840570 11427 raft_consensus.cc:2229] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:46.840484 11190 tablet_service.cc:1514] Processing DeleteTablet for tablet a9c47f22f65c40c9b7fc73353e294705 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:46 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36266
I20260809 05:35:46.840931 11427 raft_consensus.cc:2258] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:46.842209 11427 ts_tablet_manager.cc:1905] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:46.842376 11426 ts_tablet_manager.cc:1905] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:46.850150 11427 ts_tablet_manager.cc:1918] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:35:46.850462 11427 log.cc:1196] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/a9c47f22f65c40c9b7fc73353e294705
I20260809 05:35:46.851060 11427 ts_tablet_manager.cc:1939] T a9c47f22f65c40c9b7fc73353e294705 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting consensus metadata
I20260809 05:35:46.852589 10525 catalog_manager.cc:4841] TS 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233): tablet a9c47f22f65c40c9b7fc73353e294705 (table test-table [id=43866376e8994397a9fd17f9adc959c7]) successfully deleted
I20260809 05:35:46.854823 11426 ts_tablet_manager.cc:1918] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:35:46.855517 11426 log.cc:1196] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/a9c47f22f65c40c9b7fc73353e294705
I20260809 05:35:46.855525 11429 tablet_replica.cc:330] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: stopping tablet replica
I20260809 05:35:46.855746 11429 raft_consensus.cc:2229] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:46.856201 11426 ts_tablet_manager.cc:1939] T a9c47f22f65c40c9b7fc73353e294705 P b08e57effb7d4088911944f477594133: Deleting consensus metadata
I20260809 05:35:46.856426 11429 raft_consensus.cc:2258] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:46.857487 10530 catalog_manager.cc:4841] TS b08e57effb7d4088911944f477594133 (127.30.137.193:46281): tablet a9c47f22f65c40c9b7fc73353e294705 (table test-table [id=43866376e8994397a9fd17f9adc959c7]) successfully deleted
I20260809 05:35:46.857690 11429 ts_tablet_manager.cc:1905] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:46.866253 11429 ts_tablet_manager.cc:1918] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:35:46.866433 11429 log.cc:1196] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/a9c47f22f65c40c9b7fc73353e294705
I20260809 05:35:46.866986 11429 ts_tablet_manager.cc:1939] T a9c47f22f65c40c9b7fc73353e294705 P d41bfb18c8754247bc6d973ec25b42cd: Deleting consensus metadata
I20260809 05:35:46.868136 10526 catalog_manager.cc:4841] TS d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513): tablet a9c47f22f65c40c9b7fc73353e294705 (table test-table [id=43866376e8994397a9fd17f9adc959c7]) successfully deleted
Aug 09 05:35:46 dist-test-slave-3kk6 krb5kdc[10480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 05:35:46 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:46 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 05:35:46.921525 11104 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:46.909730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.30.137.254:44221)
0809 05:35:46.909959 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.909969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.910569 (+ 600us) server_negotiation.cc:407] Connection header received
0809 05:35:46.910936 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.910942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.911065 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.911189 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:46.912824 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.913949 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.915421 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.915919 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.916065 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.921227 (+ 5162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53478: BlockingRecv error: recv got EOF from 127.0.0.1:53478 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 05:35:46 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM
I20260809 05:35:46.941283 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:46.925970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53488 (local address 127.30.137.254:44221)
0809 05:35:46.926208 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:46.926218 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:46.926846 (+ 628us) server_negotiation.cc:407] Connection header received
0809 05:35:46.927108 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:46.927119 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:46.927206 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:46.927323 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:46.928604 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.929402 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.930710 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:46.931158 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:46.931307 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:46.935405 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:46.935410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:46.935414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:46.935454 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:46.938325 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:46.939134 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:46.939138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:46.939141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:46.939269 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:46.939694 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:46.939698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:46.939700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:46.939997 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:46.940348 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:46.940552 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 05:35:46.940813 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:35:46.945219 10546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:46.945904 10546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:46.953707 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 3452eb111caf49bea0b359c1f8c39908 (DEFAULT_TABLE table=test-table [id=45c35f9c0a47448486797452e1f3f53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:46.953898 10752 tablet_service.cc:1467] Processing CreateTablet for tablet 3452eb111caf49bea0b359c1f8c39908 (DEFAULT_TABLE table=test-table [id=45c35f9c0a47448486797452e1f3f53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:46.954265 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3452eb111caf49bea0b359c1f8c39908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.954324 10752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3452eb111caf49bea0b359c1f8c39908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.954388 11190 tablet_service.cc:1467] Processing CreateTablet for tablet 3452eb111caf49bea0b359c1f8c39908 (DEFAULT_TABLE table=test-table [id=45c35f9c0a47448486797452e1f3f53d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:46.954864 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3452eb111caf49bea0b359c1f8c39908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:46.967144 11451 tablet_bootstrap.cc:492] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Bootstrap starting.
I20260809 05:35:46.969139 11450 tablet_bootstrap.cc:492] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Bootstrap starting.
I20260809 05:35:46.970264 11451 tablet_bootstrap.cc:654] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.970961 11450 tablet_bootstrap.cc:654] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.972098 11452 tablet_bootstrap.cc:492] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Bootstrap starting.
I20260809 05:35:46.973845 11452 tablet_bootstrap.cc:654] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:46.975339 11451 tablet_bootstrap.cc:492] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: No bootstrap required, opened a new log
I20260809 05:35:46.975531 11451 ts_tablet_manager.cc:1397] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 05:35:46.976392 11451 raft_consensus.cc:348] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.976691 11451 raft_consensus.cc:374] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.976815 11451 raft_consensus.cc:729] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285fecbb8f0c4d30b5e1f12d1f743b53, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.977203 11451 consensus_queue.cc:260] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.979269 11451 ts_tablet_manager.cc:1428] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 05:35:46.983078 11450 tablet_bootstrap.cc:492] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: No bootstrap required, opened a new log
I20260809 05:35:46.983258 11450 ts_tablet_manager.cc:1397] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260809 05:35:46.983363 11452 tablet_bootstrap.cc:492] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: No bootstrap required, opened a new log
I20260809 05:35:46.983543 11452 ts_tablet_manager.cc:1397] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260809 05:35:46.984149 11450 raft_consensus.cc:348] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.984462 11450 raft_consensus.cc:374] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.984558 11450 raft_consensus.cc:729] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b08e57effb7d4088911944f477594133, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.984432 11452 raft_consensus.cc:348] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.984737 11452 raft_consensus.cc:374] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:46.984863 11452 raft_consensus.cc:729] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Initialized, Role: FOLLOWER
I20260809 05:35:46.984797 11450 consensus_queue.cc:260] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.985131 11452 consensus_queue.cc:260] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:46.985494 11450 ts_tablet_manager.cc:1428] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 05:35:46.985738 11452 ts_tablet_manager.cc:1428] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 05:35:47.041522 11425 raft_consensus.cc:482] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:47.041838 11425 raft_consensus.cc:504] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.043009 11425 leader_election.cc:288] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:47.043632 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3452eb111caf49bea0b359c1f8c39908" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" is_pre_election: true
I20260809 05:35:47.043674 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3452eb111caf49bea0b359c1f8c39908" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133" is_pre_election: true
I20260809 05:35:47.043923 10997 raft_consensus.cc:2454] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 0.
I20260809 05:35:47.043967 10784 raft_consensus.cc:2454] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 0.
I20260809 05:35:47.044461 11096 leader_election.cc:302] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:47.044783 11425 raft_consensus.cc:2781] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:47.044958 11425 raft_consensus.cc:482] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:47.045091 11425 raft_consensus.cc:3037] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:47.048693 11425 raft_consensus.cc:504] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.049939 11425 leader_election.cc:288] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 election: Requested vote from peers b08e57effb7d4088911944f477594133 (127.30.137.193:46281), 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233)
I20260809 05:35:47.049954 10784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3452eb111caf49bea0b359c1f8c39908" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b08e57effb7d4088911944f477594133"
I20260809 05:35:47.050200 10784 raft_consensus.cc:3037] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:47.050587 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3452eb111caf49bea0b359c1f8c39908" candidate_uuid: "d41bfb18c8754247bc6d973ec25b42cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
I20260809 05:35:47.050841 10997 raft_consensus.cc:3037] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:47.052506 10784 raft_consensus.cc:2454] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 1.
I20260809 05:35:47.053069 11096 leader_election.cc:302] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:47.053413 11425 raft_consensus.cc:2781] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:47.053692 11425 raft_consensus.cc:686] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 1 LEADER]: Becoming Leader. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Running, Role: LEADER
I20260809 05:35:47.054085 10997 raft_consensus.cc:2454] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41bfb18c8754247bc6d973ec25b42cd in term 1.
I20260809 05:35:47.054034 11425 consensus_queue.cc:237] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.057322 10548 catalog_manager.cc:5495] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195). New cstate: current_term: 1 leader_uuid: "d41bfb18c8754247bc6d973ec25b42cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } health_report { overall_health: HEALTHY } } }
I20260809 05:35:47.076848 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.071182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.30.137.195:41513)
0809 05:35:47.071401 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.071407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.071432 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:35:47.071664 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.071670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.071769 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.071885 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:47.072227 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.073392 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.074713 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.075492 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.075629 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.076045 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:47.076203 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:47.076298 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.076436 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.076527 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:35:47.080212 10997 raft_consensus.cc:1261] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Refusing update from remote peer d41bfb18c8754247bc6d973ec25b42cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:47.080272 10784 raft_consensus.cc:1261] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Refusing update from remote peer d41bfb18c8754247bc6d973ec25b42cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:47.080917 11425 consensus_queue.cc:1035] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:47.081290 11383 consensus_queue.cc:1035] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:47.093500 10546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:47.093792 10546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:47.097294 10546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Sending DeleteTablet for 3 replicas of tablet 3452eb111caf49bea0b359c1f8c39908
I20260809 05:35:47.098212 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 3452eb111caf49bea0b359c1f8c39908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:47 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:37098
I20260809 05:35:47.098805 11427 tablet_replica.cc:330] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: stopping tablet replica
I20260809 05:35:47.099045 11427 raft_consensus.cc:2229] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:47.099278 11427 raft_consensus.cc:2258] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:47.099597 10752 tablet_service.cc:1514] Processing DeleteTablet for tablet 3452eb111caf49bea0b359c1f8c39908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:47 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:39058
I20260809 05:35:47.099905 11426 tablet_replica.cc:330] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: stopping tablet replica
I20260809 05:35:47.100505 10546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:35:47.101346 10546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:35:47.101745 11426 raft_consensus.cc:2229] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:47.102525 11426 raft_consensus.cc:2258] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:47.102638 11190 tablet_service.cc:1514] Processing DeleteTablet for tablet 3452eb111caf49bea0b359c1f8c39908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:47 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36266
I20260809 05:35:47.102994 11429 tablet_replica.cc:330] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: stopping tablet replica
I20260809 05:35:47.103215 11429 raft_consensus.cc:2229] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 1 LEADER]: Raft consensus shutting down.
W20260809 05:35:47.103470 11383 consensus_queue.cc:1159] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b08e57effb7d4088911944f477594133" responder_term: 1 status { last_received { term: 1 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 1 index: 2 } }
I20260809 05:35:47.104008 11429 raft_consensus.cc:2258] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:47.105190 11426 ts_tablet_manager.cc:1905] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:47.107496 11427 ts_tablet_manager.cc:1905] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:47.109030 11429 ts_tablet_manager.cc:1905] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:47.118533 10752 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0c172ca2fb48ee861432f22bbde754 (DEFAULT_TABLE table=test-table [id=6293ac407b604ab280bd0a486bfc4952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:47.119131 10752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0c172ca2fb48ee861432f22bbde754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:47.121409 11426 ts_tablet_manager.cc:1918] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:47.121587 11426 log.cc:1196] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/3452eb111caf49bea0b359c1f8c39908
I20260809 05:35:47.114645 11427 ts_tablet_manager.cc:1918] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:47.122223 11427 log.cc:1196] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/3452eb111caf49bea0b359c1f8c39908
I20260809 05:35:47.122452 11190 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0c172ca2fb48ee861432f22bbde754 (DEFAULT_TABLE table=test-table [id=6293ac407b604ab280bd0a486bfc4952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:47.122720 11427 ts_tablet_manager.cc:1939] T 3452eb111caf49bea0b359c1f8c39908 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting consensus metadata
I20260809 05:35:47.123015 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0c172ca2fb48ee861432f22bbde754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:47.123176 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0c172ca2fb48ee861432f22bbde754 (DEFAULT_TABLE table=test-table [id=6293ac407b604ab280bd0a486bfc4952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:35:47.123571 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b0c172ca2fb48ee861432f22bbde754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:47.127637 11450 tablet_bootstrap.cc:492] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Bootstrap starting.
I20260809 05:35:47.129199 11429 ts_tablet_manager.cc:1918] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:35:47.129379 11450 tablet_bootstrap.cc:654] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:47.129508 11429 log.cc:1196] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/3452eb111caf49bea0b359c1f8c39908
I20260809 05:35:47.130117 11429 ts_tablet_manager.cc:1939] T 3452eb111caf49bea0b359c1f8c39908 P d41bfb18c8754247bc6d973ec25b42cd: Deleting consensus metadata
I20260809 05:35:47.130439 11452 tablet_bootstrap.cc:492] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Bootstrap starting.
I20260809 05:35:47.131460 10526 catalog_manager.cc:4841] TS d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513): tablet 3452eb111caf49bea0b359c1f8c39908 (table test-table [id=45c35f9c0a47448486797452e1f3f53d]) successfully deleted
I20260809 05:35:47.132346 11452 tablet_bootstrap.cc:654] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:47.134014 11451 tablet_bootstrap.cc:492] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Bootstrap starting.
I20260809 05:35:47.135634 11452 tablet_bootstrap.cc:492] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: No bootstrap required, opened a new log
I20260809 05:35:47.135635 11450 tablet_bootstrap.cc:492] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: No bootstrap required, opened a new log
I20260809 05:35:47.135814 11451 tablet_bootstrap.cc:654] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:47.135830 11452 ts_tablet_manager.cc:1397] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260809 05:35:47.135957 11450 ts_tablet_manager.cc:1397] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 05:35:47.136742 11450 raft_consensus.cc:348] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.136739 11452 raft_consensus.cc:348] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.137151 11450 raft_consensus.cc:374] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:47.137151 11452 raft_consensus.cc:374] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:47.137252 11452 raft_consensus.cc:729] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41bfb18c8754247bc6d973ec25b42cd, State: Initialized, Role: FOLLOWER
I20260809 05:35:47.137285 11450 raft_consensus.cc:729] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b08e57effb7d4088911944f477594133, State: Initialized, Role: FOLLOWER
I20260809 05:35:47.137639 11450 consensus_queue.cc:260] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.137645 11452 consensus_queue.cc:260] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.139427 11450 ts_tablet_manager.cc:1428] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 05:35:47.141518 11426 ts_tablet_manager.cc:1939] T 3452eb111caf49bea0b359c1f8c39908 P b08e57effb7d4088911944f477594133: Deleting consensus metadata
I20260809 05:35:47.141732 10525 catalog_manager.cc:4841] TS 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233): tablet 3452eb111caf49bea0b359c1f8c39908 (table test-table [id=45c35f9c0a47448486797452e1f3f53d]) successfully deleted
I20260809 05:35:47.143316 11452 ts_tablet_manager.cc:1428] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260809 05:35:47.143741 10530 catalog_manager.cc:4841] TS b08e57effb7d4088911944f477594133 (127.30.137.193:46281): tablet 3452eb111caf49bea0b359c1f8c39908 (table test-table [id=45c35f9c0a47448486797452e1f3f53d]) successfully deleted
I20260809 05:35:47.146795 11451 tablet_bootstrap.cc:492] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: No bootstrap required, opened a new log
I20260809 05:35:47.147004 11451 ts_tablet_manager.cc:1397] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260809 05:35:47.147831 11451 raft_consensus.cc:348] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.148131 11451 raft_consensus.cc:374] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:47.148218 11451 raft_consensus.cc:729] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285fecbb8f0c4d30b5e1f12d1f743b53, State: Initialized, Role: FOLLOWER
I20260809 05:35:47.148466 11451 consensus_queue.cc:260] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.149060 11451 ts_tablet_manager.cc:1428] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 05:35:47.347695 11339 raft_consensus.cc:482] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:35:47.348011 11339 raft_consensus.cc:504] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.349575 11339 leader_election.cc:288] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233), d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513)
I20260809 05:35:47.359647 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.350021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:53281 (local address 127.30.137.195:41513)
0809 05:35:47.350293 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.350302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.351258 (+ 956us) server_negotiation.cc:407] Connection header received
0809 05:35:47.351540 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.351549 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.351682 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.351801 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:47.352791 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.353929 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.356914 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.358438 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.358626 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.358756 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.359158 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.359321 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:35:47.360394 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.350325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:53937 (local address 127.30.137.194:42233)
0809 05:35:47.351518 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.351522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.351538 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:35:47.351604 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.351608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.351682 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.351797 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:35:47.352501 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.353949 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.356992 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.358448 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.358626 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.358741 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.360033 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.360155 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048,"thread_start_us":937,"threads_started":1}
I20260809 05:35:47.360669 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.349880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:42233 (local address 127.30.137.193:53937)
0809 05:35:47.350855 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:47.350883 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:47.351049 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:47.351819 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:47.351824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:47.351850 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:47.352325 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.352337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.354107 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.354113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.356824 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.356843 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.358905 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.358910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.359036 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.359059 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:35:47.359187 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 05:35:47.359322 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":97,"thread_start_us":943,"threads_started":1}
I20260809 05:35:47.360707 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.349928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:41513 (local address 127.30.137.193:53281)
0809 05:35:47.351137 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:47.351155 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:47.351359 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:47.352001 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:47.352012 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:47.352041 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:35:47.352570 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.352583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.354107 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.354113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.356662 (+ 2549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.356685 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.358732 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.358741 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.358952 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.358990 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:35:47.359109 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 05:35:47.359337 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":512,"thread_start_us":537,"threads_started":1}
I20260809 05:35:47.361451 11219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0c172ca2fb48ee861432f22bbde754" candidate_uuid: "b08e57effb7d4088911944f477594133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d41bfb18c8754247bc6d973ec25b42cd" is_pre_election: true
I20260809 05:35:47.361819 11219 raft_consensus.cc:2454] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b08e57effb7d4088911944f477594133 in term 0.
I20260809 05:35:47.361760 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0c172ca2fb48ee861432f22bbde754" candidate_uuid: "b08e57effb7d4088911944f477594133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" is_pre_election: true
I20260809 05:35:47.362003 10997 raft_consensus.cc:2454] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b08e57effb7d4088911944f477594133 in term 0.
I20260809 05:35:47.362447 10677 leader_election.cc:302] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:47.362926 11339 raft_consensus.cc:2781] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:35:47.363075 11339 raft_consensus.cc:482] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:35:47.363191 11339 raft_consensus.cc:3037] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:47.365835 11339 raft_consensus.cc:504] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.366554 11339 leader_election.cc:288] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [CANDIDATE]: Term 1 election: Requested vote from peers 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233), d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513)
I20260809 05:35:47.367141 11219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0c172ca2fb48ee861432f22bbde754" candidate_uuid: "b08e57effb7d4088911944f477594133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d41bfb18c8754247bc6d973ec25b42cd"
I20260809 05:35:47.367166 10997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b0c172ca2fb48ee861432f22bbde754" candidate_uuid: "b08e57effb7d4088911944f477594133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53"
I20260809 05:35:47.367337 11219 raft_consensus.cc:3037] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:47.367411 10997 raft_consensus.cc:3037] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:47.369547 11219 raft_consensus.cc:2454] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b08e57effb7d4088911944f477594133 in term 1.
I20260809 05:35:47.370059 10677 leader_election.cc:302] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b08e57effb7d4088911944f477594133, d41bfb18c8754247bc6d973ec25b42cd; no voters:
I20260809 05:35:47.370424 11339 raft_consensus.cc:2781] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:47.370503 10997 raft_consensus.cc:2454] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b08e57effb7d4088911944f477594133 in term 1.
I20260809 05:35:47.371068 11339 raft_consensus.cc:686] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 1 LEADER]: Becoming Leader. State: Replica: b08e57effb7d4088911944f477594133, State: Running, Role: LEADER
I20260809 05:35:47.371519 11339 consensus_queue.cc:237] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } }
I20260809 05:35:47.375785 10546 catalog_manager.cc:5495] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 reported cstate change: term changed from 0 to 1, leader changed from <none> to b08e57effb7d4088911944f477594133 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "b08e57effb7d4088911944f477594133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b08e57effb7d4088911944f477594133" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 } health_report { overall_health: UNKNOWN } } }
I20260809 05:35:47.433676 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.427498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.30.137.193:46281)
0809 05:35:47.428134 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.428138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.428158 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:35:47.428244 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.428247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.428331 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.428517 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:47.428984 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.429837 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.431280 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.431731 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.431884 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.432386 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:47.432982 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:47.433144 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.433228 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.433316 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":369,"threads_started":1}
I20260809 05:35:47.452657 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.445704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:34543 (local address 127.30.137.195:41513)
0809 05:35:47.445942 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.445948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.446435 (+ 487us) server_negotiation.cc:407] Connection header received
0809 05:35:47.446702 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.446707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.446792 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.446881 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:47.447591 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.448349 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.450503 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.451086 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.451193 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.451806 (+ 613us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:47.451922 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:47.451990 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.452216 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.452287 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 05:35:47.452795 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.445459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:41513 (local address 127.30.137.193:34543)
0809 05:35:47.446301 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:47.446324 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:47.446509 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:47.446935 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:47.446940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:47.446957 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:35:47.447401 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.447417 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.448518 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.448525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.450319 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.450337 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.451303 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.451311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.451506 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.451597 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:35:47.452034 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:35:47.452047 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 05:35:47.452165 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 05:35:47.452334 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":498,"threads_started":1}
I20260809 05:35:47.463201 11219 raft_consensus.cc:1261] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Refusing update from remote peer b08e57effb7d4088911944f477594133: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:47.463312 10997 raft_consensus.cc:1261] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Refusing update from remote peer b08e57effb7d4088911944f477594133: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:35:47.464119 11339 consensus_queue.cc:1035] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d41bfb18c8754247bc6d973ec25b42cd" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 41513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:47.464525 11470 consensus_queue.cc:1035] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [LEADER]: Connected to new peer: Peer: permanent_uuid: "285fecbb8f0c4d30b5e1f12d1f743b53" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 42233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:35:47.505409 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.498976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:42385 (local address 127.30.137.193:46281)
0809 05:35:47.499195 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.499201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.499551 (+ 350us) server_negotiation.cc:407] Connection header received
0809 05:35:47.499826 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.499832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.499918 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.500017 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:35:47.500591 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.501408 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.503506 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.504033 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.504188 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.504634 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:35:47.504739 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:35:47.504839 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.505009 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.505091 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:35:47.505565 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.498529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:46281 (local address 127.30.137.195:42385)
0809 05:35:47.499274 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:47.499294 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:47.499644 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:47.500038 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:47.500042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:47.500054 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 05:35:47.500424 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.500434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.501588 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.501592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.503319 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:47.503336 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.504229 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.504237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:47.504416 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.504455 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 05:35:47.504850 (+ 395us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 05:35:47.504879 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 05:35:47.504982 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 05:35:47.505132 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":780,"threads_started":1}
I20260809 05:35:47.547334 10546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:47.547674 10546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:35:47.551676 10546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a6238bd708d4cb789cf8159dff136cf: Sending DeleteTablet for 3 replicas of tablet 5b0c172ca2fb48ee861432f22bbde754
I20260809 05:35:47.552675 10752 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0c172ca2fb48ee861432f22bbde754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:47 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:39058
I20260809 05:35:47.553097 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0c172ca2fb48ee861432f22bbde754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:47 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:37098
I20260809 05:35:47.553442 11426 tablet_replica.cc:330] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: stopping tablet replica
I20260809 05:35:47.553473 11190 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b0c172ca2fb48ee861432f22bbde754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:35:47 UTC) from {username='oryx', principal='oryx/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36266
I20260809 05:35:47.553638 11426 raft_consensus.cc:2229] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:35:47.553838 11429 tablet_replica.cc:330] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: stopping tablet replica
I20260809 05:35:47.554039 11429 raft_consensus.cc:2229] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:47.554214 11426 raft_consensus.cc:2258] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:47.554267 11429 raft_consensus.cc:2258] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:47.554595 11427 tablet_replica.cc:330] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: stopping tablet replica
I20260809 05:35:47.554813 11427 raft_consensus.cc:2229] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:35:47.555042 11427 raft_consensus.cc:2258] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:35:47.556501 11429 ts_tablet_manager.cc:1905] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:47.557108 11427 ts_tablet_manager.cc:1905] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:47.557385 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 10651
I20260809 05:35:47.563872 11426 ts_tablet_manager.cc:1905] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:35:47.571781 11426 ts_tablet_manager.cc:1918] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:35:47.573973 11427 ts_tablet_manager.cc:1918] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:35:47.574162 11429 ts_tablet_manager.cc:1918] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 05:35:47.574265 11427 log.cc:1196] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/5b0c172ca2fb48ee861432f22bbde754
I20260809 05:35:47.574368 11429 log.cc:1196] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/5b0c172ca2fb48ee861432f22bbde754
I20260809 05:35:47.575026 11427 ts_tablet_manager.cc:1939] T 5b0c172ca2fb48ee861432f22bbde754 P 285fecbb8f0c4d30b5e1f12d1f743b53: Deleting consensus metadata
I20260809 05:35:47.575242 11426 log.cc:1196] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/5b0c172ca2fb48ee861432f22bbde754
I20260809 05:35:47.575678 11429 ts_tablet_manager.cc:1939] T 5b0c172ca2fb48ee861432f22bbde754 P d41bfb18c8754247bc6d973ec25b42cd: Deleting consensus metadata
I20260809 05:35:47.575855 11426 ts_tablet_manager.cc:1939] T 5b0c172ca2fb48ee861432f22bbde754 P b08e57effb7d4088911944f477594133: Deleting consensus metadata
I20260809 05:35:47.577164 10525 catalog_manager.cc:4841] TS 285fecbb8f0c4d30b5e1f12d1f743b53 (127.30.137.194:42233): tablet 5b0c172ca2fb48ee861432f22bbde754 (table test-table [id=6293ac407b604ab280bd0a486bfc4952]) successfully deleted
I20260809 05:35:47.577167 10526 catalog_manager.cc:4841] TS d41bfb18c8754247bc6d973ec25b42cd (127.30.137.195:41513): tablet 5b0c172ca2fb48ee861432f22bbde754 (table test-table [id=6293ac407b604ab280bd0a486bfc4952]) successfully deleted
I20260809 05:35:47.578140 10530 catalog_manager.cc:4841] TS b08e57effb7d4088911944f477594133 (127.30.137.193:46281): tablet 5b0c172ca2fb48ee861432f22bbde754 (table test-table [id=6293ac407b604ab280bd0a486bfc4952]) successfully deleted
Aug 09 05:35:47 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.193@KRBTEST.COM
I20260809 05:35:47.602362 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.559643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.30.137.193:46281)
0809 05:35:47.559842 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.559849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.579098 (+ 19249us) server_negotiation.cc:407] Connection header received
0809 05:35:47.579324 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.579331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.579438 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.579596 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:47.581297 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.582745 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.584705 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.585542 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.585715 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.591903 (+ 6188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:47.591934 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:47.591956 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:47.592024 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:47.597058 (+ 5034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:47.598458 (+ 1400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:47.598470 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:47.598475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:47.598758 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:47.599329 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:47.599334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:47.599337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:47.599865 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:47.600233 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.601575 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.601853 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 05:35:47.603991 10849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:47.720577 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 10651
I20260809 05:35:47.746230 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 10866
Aug 09 05:35:47 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.194@KRBTEST.COM
I20260809 05:35:47.762411 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.747144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.30.137.194:42233)
0809 05:35:47.747368 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.747374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.747397 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:35:47.747628 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.747632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.747707 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.747821 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:47.749133 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.749929 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.751530 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.751966 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.752166 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.756668 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:47.756688 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:47.756704 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:47.756760 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:47.759917 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:47.760608 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:47.760616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:47.760618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:47.760751 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:47.761087 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:47.761090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:47.761092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:47.761464 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:47.761694 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.761947 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.762149 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:35:47.763185 11057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:47.892580 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 10866
I20260809 05:35:47.925205 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 11070
Aug 09 05:35:47 dist-test-slave-3kk6 krb5kdc[10480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.137.195@KRBTEST.COM
I20260809 05:35:47.941847 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:47.926050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.30.137.195:41513)
0809 05:35:47.926247 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:47.926252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:47.926272 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:35:47.926438 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:47.926442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:47.926518 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:47.926604 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:47.927985 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.928773 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.930031 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:47.930826 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:47.930961 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:47.935970 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:47.935990 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:47.936007 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:47.936049 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:47.939007 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:47.939718 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:47.939726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:47.939728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:47.939911 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:47.940247 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:47.940250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:47.940252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:47.940706 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:47.940959 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:47.941228 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 05:35:47.941449 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:35:47.942539 11275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:48.094884 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 11070
I20260809 05:35:48.131505 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 10502
I20260809 05:35:48.132833 10609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:48.255445 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 10502
2026-08-09T05:35:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6436 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:48 dist-test-slave-3kk6 krb5kdc[11503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:48 dist-test-slave-3kk6 krb5kdc[11503](info): set up 2 sockets
Aug 09 05:35:48 dist-test-slave-3kk6 krb5kdc[11503](info): commencing operation
krb5kdc: starting...
W20260809 05:35:49.588719 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:49 dist-test-slave-3kk6 krb5kdc[11503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:49.890575 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44647
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:42221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:44647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:50.248265 11517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:50.248587 11517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:50.248740 11517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:50.256312 11517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:50.256451 11517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:50.256559 11517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:50.256662 11517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:50.256763 11517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:50.267071 11517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:42221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:44647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:50.268484 11517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:50.269886 11517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:50.282397 11530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:50.282321 11528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:50.282929 11525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:50.283528 11517 server_base.cc:1034] running on GCE node
I20260809 05:35:50.284359 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:50.285900 11517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:50.287139 11517 hybrid_clock.cc:648] HybridClock initialized: now 1786253750287086 us; error 69 us; skew 500 ppm
I20260809 05:35:50.287505 11517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T05:35:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2102 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:35:50 dist-test-slave-3kk6 krb5kdc[11538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:35:50 dist-test-slave-3kk6 krb5kdc[11538](info): set up 2 sockets
Aug 09 05:35:50 dist-test-slave-3kk6 krb5kdc[11538](info): commencing operation
krb5kdc: starting...
W20260809 05:35:51.717267 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:35:51 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:35:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:35:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:35:52.013942 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35515
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:52.367254 11559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:52.367626 11559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:52.367820 11559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:52.375437 11559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:52.375573 11559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:52.375664 11559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:52.375751 11559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:52.375823 11559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:52.386308 11559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:52.387832 11559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:52.389339 11559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:52.402854 11569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:52.402937 11574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:52.402889 11570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:52.403849 11559 server_base.cc:1034] running on GCE node
I20260809 05:35:52.404732 11559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:52.406292 11559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:52.407539 11559 hybrid_clock.cc:648] HybridClock initialized: now 1786253752407496 us; error 60 us; skew 500 ppm
I20260809 05:35:52.407953 11559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:52 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253752, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:52.411562 11559 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:52.413416 11559 webserver.cc:460] Webserver started at http://127.30.137.254:44299/ using document root <none> and password file <none>
I20260809 05:35:52.414129 11559 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:52.414233 11559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:52.414510 11559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:52.417678 11559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "eef02030514a4316a3ef1d3c853ee263"
format_stamp: "Formatted at 2026-08-09 05:35:52 on dist-test-slave-3kk6"
server_key: "339c1efa389edd573cbb5e02e1f6334b"
server_key_iv: "33f1b3a68d182b5cc8135db3cc2f9b65"
server_key_version: "encryptionkey@0"
I20260809 05:35:52.418434 11559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "eef02030514a4316a3ef1d3c853ee263"
format_stamp: "Formatted at 2026-08-09 05:35:52 on dist-test-slave-3kk6"
server_key: "339c1efa389edd573cbb5e02e1f6334b"
server_key_iv: "33f1b3a68d182b5cc8135db3cc2f9b65"
server_key_version: "encryptionkey@0"
I20260809 05:35:52.423895 11559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 05:35:52.427507 11584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:52.428990 11559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:52.429201 11559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "eef02030514a4316a3ef1d3c853ee263"
format_stamp: "Formatted at 2026-08-09 05:35:52 on dist-test-slave-3kk6"
server_key: "339c1efa389edd573cbb5e02e1f6334b"
server_key_iv: "33f1b3a68d182b5cc8135db3cc2f9b65"
server_key_version: "encryptionkey@0"
I20260809 05:35:52.429419 11559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:52.448594 11559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:52.453380 11559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:52.453711 11559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:52.473481 11559 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35515
I20260809 05:35:52.473512 11657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35515 every 8 connection(s)
I20260809 05:35:52.475116 11559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:52.478924 11658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:35:52.482898 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 11559
I20260809 05:35:52.483150 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:35:52.484059 31271 external_mini_cluster.cc:1428] Setting key 19b634d012b4f77d16917428cbdc1961
I20260809 05:35:52.493460 11658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Bootstrap starting.
I20260809 05:35:52.497596 11658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Neither blocks nor log segments found. Creating new log.
I20260809 05:35:52.499051 11658 log.cc:824] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:35:52 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:52.502300 11658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: No bootstrap required, opened a new log
I20260809 05:35:52.508893 11665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:52.486400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.30.137.254:35515)
0809 05:35:52.487277 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:52.487289 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:52.487334 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:35:52.489029 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:52.489054 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:52.489493 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:52.489860 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:52.491861 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:52.493230 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:52.495065 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:52.495785 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:52.495932 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:52.501128 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:52.501137 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:52.501152 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:52.501187 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:52.504563 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:52.505480 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:52.505487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:52.505493 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:52.505674 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:52.506085 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:52.506088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:52.506090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:52.506628 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:52.506947 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:52.507389 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 05:35:52.507822 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":429,"threads_started":1}
I20260809 05:35:52.512006 11658 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:52.512506 11658 raft_consensus.cc:374] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:35:52.512624 11658 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef02030514a4316a3ef1d3c853ee263, State: Initialized, Role: FOLLOWER
I20260809 05:35:52.513242 11658 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:52.513489 11658 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:52.513643 11658 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:52.513830 11658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:35:52.516480 11658 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:52.516989 11658 leader_election.cc:302] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef02030514a4316a3ef1d3c853ee263; no voters:
I20260809 05:35:52.517593 11658 leader_election.cc:288] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:35:52.517735 11670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:35:52.519237 11670 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 LEADER]: Becoming Leader. State: Replica: eef02030514a4316a3ef1d3c853ee263, State: Running, Role: LEADER
I20260809 05:35:52.519994 11658 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:52.519958 11670 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:52.523173 11671 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eef02030514a4316a3ef1d3c853ee263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } } }
I20260809 05:35:52.523147 11672 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef02030514a4316a3ef1d3c853ee263. Latest consensus state: current_term: 1 leader_uuid: "eef02030514a4316a3ef1d3c853ee263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } } }
I20260809 05:35:52.523689 11671 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:52.523767 11672 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:52.524643 11684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:52.535264 11684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:52.546526 11684 catalog_manager.cc:1347] Generated new cluster ID: 868eacfb44ed40f5becfce85dfb80826
I20260809 05:35:52.546701 11684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:52.564306 11684 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:35:52.565742 11684 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:52.571993 11684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Generated new TSK 0
I20260809 05:35:52.572826 11684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:35:52.656358 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35515
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:53.032068 11699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:53.032424 11699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:53.032608 11699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:53.040144 11699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:53.040318 11699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:53.040539 11699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:53.050597 11699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:53.051985 11699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:53.053297 11699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:53.066923 11706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:53.066820 11708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:53.066841 11710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:53.067816 11699 server_base.cc:1034] running on GCE node
I20260809 05:35:53.068405 11699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:53.069360 11699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:53.070574 11699 hybrid_clock.cc:648] HybridClock initialized: now 1786253753070511 us; error 51 us; skew 500 ppm
I20260809 05:35:53.070956 11699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:53 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:53.074538 11699 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:35:53.076113 11699 webserver.cc:460] Webserver started at http://127.30.137.193:39701/ using document root <none> and password file <none>
I20260809 05:35:53.076879 11699 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:53.077036 11699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:53.077374 11699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:53.080628 11699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "8b82d68f573d4d9f824a66ecf302aaec"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "467d11ae8b1bf53081c2bd2cf7d98c94"
server_key_iv: "769874e7c33a6adf545581f04f621330"
server_key_version: "encryptionkey@0"
I20260809 05:35:53.081709 11699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "8b82d68f573d4d9f824a66ecf302aaec"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "467d11ae8b1bf53081c2bd2cf7d98c94"
server_key_iv: "769874e7c33a6adf545581f04f621330"
server_key_version: "encryptionkey@0"
I20260809 05:35:53.088025 11699 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 05:35:53.092270 11720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:53.093652 11699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:53.093902 11699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "8b82d68f573d4d9f824a66ecf302aaec"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "467d11ae8b1bf53081c2bd2cf7d98c94"
server_key_iv: "769874e7c33a6adf545581f04f621330"
server_key_version: "encryptionkey@0"
I20260809 05:35:53.094134 11699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:53.115527 11699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:53.120347 11699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:53.120741 11699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:53.121853 11699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:53.123468 11699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:53.123575 11699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:53.123744 11699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:53.123821 11699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:53.159572 11699 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44305
I20260809 05:35:53.159636 11897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44305 every 8 connection(s)
I20260809 05:35:53.161043 11699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:53.161913 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 11699
I20260809 05:35:53.162166 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:35:53.163041 31271 external_mini_cluster.cc:1428] Setting key 6c573b84a131df1aabe89706ddf3a6be
Aug 09 05:35:53 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:53.186926 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:53.164269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:32949 (local address 127.30.137.254:35515)
0809 05:35:53.164746 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:53.164751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:53.166045 (+ 1294us) server_negotiation.cc:407] Connection header received
0809 05:35:53.169153 (+ 3108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:53.169157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:53.169246 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:53.169349 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:53.171667 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.172413 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.174988 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.175403 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.175503 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:53.180271 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:53.180277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:53.180280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:53.180317 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:53.183210 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:53.184292 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:53.184297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:53.184299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:53.184428 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:53.185013 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:53.185016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:53.185018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:53.185298 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:53.185426 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:53.186402 (+ 976us) server_negotiation.cc:299] Negotiation successful
0809 05:35:53.186592 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":266,"threads_started":1}
I20260809 05:35:53.188823 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:53.164833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:32949)
0809 05:35:53.165838 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:53.165907 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:53.168818 (+ 2911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:53.169653 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:53.169666 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:53.170525 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:53.171451 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:53.171480 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.172582 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.172590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:53.174832 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:53.174851 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.175647 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.175656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:53.175997 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:53.176814 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:53.176856 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:53.179978 (+ 3122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:53.183406 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:53.183420 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:53.183443 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:53.184150 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:53.184546 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:53.184554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:53.184560 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:53.184884 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:53.185444 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:53.185455 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:53.185968 (+ 513us) client_negotiation.cc:769] Sending connection context
0809 05:35:53.186302 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 05:35:53.187037 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":372,"threads_started":1}
I20260809 05:35:53.190547 11898 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35515
I20260809 05:35:53.190923 11898 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:53.191725 11898 heartbeater.cc:511] Master 127.30.137.254:35515 requested a full tablet report, sending...
I20260809 05:35:53.194384 11608 ts_manager.cc:194] Registered new tserver with Master: 8b82d68f573d4d9f824a66ecf302aaec (127.30.137.193:44305)
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:35:53.196919 11608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:32949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:35:53.234638 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35515
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:53.583012 11907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:53.583384 11907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:53.583567 11907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:53.592337 11907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:53.592525 11907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:53.592715 11907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:53.603142 11907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:53.604560 11907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:53.605919 11907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:53.619215 11915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:53.619174 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:53.619251 11916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:53.620007 11907 server_base.cc:1034] running on GCE node
I20260809 05:35:53.620599 11907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:53.621598 11907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:53.622756 11907 hybrid_clock.cc:648] HybridClock initialized: now 1786253753622710 us; error 31 us; skew 500 ppm
I20260809 05:35:53.623096 11907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:53 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:53.626286 11907 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:35:53.627967 11907 webserver.cc:460] Webserver started at http://127.30.137.194:36877/ using document root <none> and password file <none>
I20260809 05:35:53.628662 11907 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:53.628822 11907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:53.629150 11907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:53.632284 11907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "4e4006e989654c0da81a9a746c860873"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "0f0bde5c7f354ddb42308659f1a257e6"
server_key_iv: "cc78f7a15ab79f90d8519f41d94882e4"
server_key_version: "encryptionkey@0"
I20260809 05:35:53.633256 11907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "4e4006e989654c0da81a9a746c860873"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "0f0bde5c7f354ddb42308659f1a257e6"
server_key_iv: "cc78f7a15ab79f90d8519f41d94882e4"
server_key_version: "encryptionkey@0"
I20260809 05:35:53.639120 11907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:35:53.642897 11931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:53.644379 11907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:35:53.644562 11907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "4e4006e989654c0da81a9a746c860873"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "0f0bde5c7f354ddb42308659f1a257e6"
server_key_iv: "cc78f7a15ab79f90d8519f41d94882e4"
server_key_version: "encryptionkey@0"
I20260809 05:35:53.644789 11907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:53.670830 11907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:53.675132 11907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:53.675467 11907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:53.676491 11907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:53.677886 11907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:53.677997 11907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:53.678167 11907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:53.678241 11907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:53.713938 11907 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:34711
I20260809 05:35:53.713997 12097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:34711 every 8 connection(s)
I20260809 05:35:53.715504 11907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:53.725630 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 11907
I20260809 05:35:53.725800 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:35:53.726280 31271 external_mini_cluster.cc:1428] Setting key 2521f476551f67f1681aac73db887dcc
Aug 09 05:35:53 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253753, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:53.737404 12100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:53.718147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:35449 (local address 127.30.137.254:35515)
0809 05:35:53.718634 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:53.718638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:53.719547 (+ 909us) server_negotiation.cc:407] Connection header received
0809 05:35:53.721637 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:53.721643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:53.721744 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:53.721862 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:53.723452 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.724278 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.725887 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.726353 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.726499 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:53.730815 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:53.730821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:53.730824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:53.730860 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:53.733917 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:53.734952 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:53.734956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:53.734959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:53.735094 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:53.735619 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:53.735622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:53.735624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:53.735902 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:53.736018 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:53.736943 (+ 925us) server_negotiation.cc:299] Negotiation successful
0809 05:35:53.737118 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":270,"threads_started":1}
I20260809 05:35:53.739037 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:53.718607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:35449)
0809 05:35:53.719380 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:53.719420 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:53.721333 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:53.722105 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:53.722114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:53.722639 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:53.723279 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:53.723298 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.724435 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.724439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:53.725688 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:53.725735 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:53.726614 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:53.726622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:53.726970 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:53.727697 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:53.727720 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:53.730505 (+ 2785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:53.734082 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:53.734093 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:53.734114 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:53.734822 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:53.735206 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:53.735214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:53.735220 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:53.735505 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:53.736038 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:53.736048 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:53.736544 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 05:35:53.736864 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 05:35:53.737488 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":278,"threads_started":1}
I20260809 05:35:53.740738 12098 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35515
I20260809 05:35:53.741078 12098 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:53.741808 12098 heartbeater.cc:511] Master 127.30.137.254:35515 requested a full tablet report, sending...
I20260809 05:35:53.743324 11608 ts_manager.cc:194] Registered new tserver with Master: 4e4006e989654c0da81a9a746c860873 (127.30.137.194:34711)
I20260809 05:35:53.744167 11608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:35449
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:35:53.790163 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35515
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:54.145341 12105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:54.145686 12105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:54.145865 12105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:54.153434 12105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:54.153620 12105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:54.153806 12105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:54.166651 12105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:54.168107 12105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:54.169438 12105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:54.183173 12111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:54.183418 12113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:54.183591 12110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:54.184203 12105 server_base.cc:1034] running on GCE node
I20260809 05:35:54.184829 12105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:54.185770 12105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:54.186978 12105 hybrid_clock.cc:648] HybridClock initialized: now 1786253754186924 us; error 43 us; skew 500 ppm
I20260809 05:35:54.187325 12105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:54 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253754, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:54.190778 12105 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:35:54.192333 12105 webserver.cc:460] Webserver started at http://127.30.137.195:40417/ using document root <none> and password file <none>
I20260809 05:35:54.193079 12105 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:54.193243 12105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:54.193609 12105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:35:54.196798 12105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "06b827e03d35496c8d1778ae6c0661af"
format_stamp: "Formatted at 2026-08-09 05:35:54 on dist-test-slave-3kk6"
server_key: "cb93563bf62d3e423cc2096b2114e067"
server_key_iv: "478f4e4dea7eaec98717ffb31a6742f1"
server_key_version: "encryptionkey@0"
I20260809 05:35:54.197674 12105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "06b827e03d35496c8d1778ae6c0661af"
format_stamp: "Formatted at 2026-08-09 05:35:54 on dist-test-slave-3kk6"
server_key: "cb93563bf62d3e423cc2096b2114e067"
server_key_iv: "478f4e4dea7eaec98717ffb31a6742f1"
server_key_version: "encryptionkey@0"
I20260809 05:35:54.200973 11898 heartbeater.cc:503] Master 127.30.137.254:35515 was elected leader, sending a full tablet report...
I20260809 05:35:54.204726 12105 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260809 05:35:54.208523 12126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:54.210041 12105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:35:54.210186 12105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "06b827e03d35496c8d1778ae6c0661af"
format_stamp: "Formatted at 2026-08-09 05:35:54 on dist-test-slave-3kk6"
server_key: "cb93563bf62d3e423cc2096b2114e067"
server_key_iv: "478f4e4dea7eaec98717ffb31a6742f1"
server_key_version: "encryptionkey@0"
I20260809 05:35:54.210384 12105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:54.228633 12105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:54.233146 12105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:54.233460 12105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:54.234514 12105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:54.235941 12105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:54.236016 12105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:54.236104 12105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:54.236150 12105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:54.271884 12105 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:35315
I20260809 05:35:54.271935 12291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:35315 every 8 connection(s)
I20260809 05:35:54.273241 12105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:54.280262 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 12105
I20260809 05:35:54.280471 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:35:54.281000 31271 external_mini_cluster.cc:1428] Setting key e1b97c11dc07146816e823410b3eca4d
Aug 09 05:35:54 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253754, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:35:54.294394 12295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:54.275985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:44077 (local address 127.30.137.254:35515)
0809 05:35:54.276482 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:54.276487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:54.277322 (+ 835us) server_negotiation.cc:407] Connection header received
0809 05:35:54.279326 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:54.279332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:54.279433 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:54.279562 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:54.281360 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.282418 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.284601 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.285035 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.285137 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:54.288818 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:54.288824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:54.288828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:54.288862 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:54.291544 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.292299 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.292304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.292306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.292425 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.292814 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.292817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.292819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.293095 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:54.293216 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:54.293910 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 05:35:54.294092 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":260,"threads_started":1}
I20260809 05:35:54.295578 12296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:54.276376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:44077)
0809 05:35:54.277131 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:54.277173 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:54.279001 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:54.279848 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:54.279858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:54.280457 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:54.281160 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:54.281188 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.282649 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.282655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:54.284444 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:54.284463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.285238 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.285243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:54.285456 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:54.286143 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:54.286171 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:54.288566 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:35:54.291702 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:54.291710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:54.291724 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:54.292181 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:54.292522 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:35:54.292525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:35:54.292528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:35:54.292708 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:35:54.293206 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:35:54.293212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:35:54.293564 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 05:35:54.293801 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 05:35:54.294223 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":305,"threads_started":1}
I20260809 05:35:54.297185 12293 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35515
I20260809 05:35:54.297544 12293 heartbeater.cc:464] Registering TS with master...
I20260809 05:35:54.298308 12293 heartbeater.cc:511] Master 127.30.137.254:35515 requested a full tablet report, sending...
I20260809 05:35:54.300029 11607 ts_manager.cc:194] Registered new tserver with Master: 06b827e03d35496c8d1778ae6c0661af (127.30.137.195:35315)
I20260809 05:35:54.300969 11607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:44077
I20260809 05:35:54.309495 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab.
Entry for principal oryx/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab.
WARNING: no policy specified for oryx/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab.
Entry for principal oryx/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab.
WARNING: no policy specified for oryx/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab.
Entry for principal oryx/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab.
I20260809 05:35:54.493364 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 11699
Aug 09 05:35:54 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:35:54.511703 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:54.494808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.30.137.193:44305)
0809 05:35:54.495621 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:54.495631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:54.495911 (+ 280us) server_negotiation.cc:407] Connection header received
0809 05:35:54.496069 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:54.496076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:54.496318 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:54.496473 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:54.497979 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.499231 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.500562 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.501194 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.501321 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:54.505665 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:54.505680 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:54.505696 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:54.505735 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:54.508707 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.509419 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.509426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.509428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.509579 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.509947 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.509950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.509952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.510352 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:54.510610 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:54.511014 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 05:35:54.511310 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":373,"threads_started":1}
I20260809 05:35:54.513108 11892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:54.599117 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 11699
I20260809 05:35:54.624276 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 11907
Aug 09 05:35:54 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:35:54.641386 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:54.625229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.30.137.194:34711)
0809 05:35:54.625863 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:54.625872 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:54.625901 (+ 29us) server_negotiation.cc:407] Connection header received
0809 05:35:54.625966 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:54.625973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:54.626185 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:54.626318 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:54.627721 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.628759 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.630092 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.630811 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.630927 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:54.635363 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:54.635376 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:54.635390 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:54.635435 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:54.638592 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.639379 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.639387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.639390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.639525 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.639851 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.639855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.639857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.640235 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:54.640490 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:54.640736 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 05:35:54.640957 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":349,"threads_started":1}
I20260809 05:35:54.642824 12090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:54.727403 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 11907
I20260809 05:35:54.752312 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12105
Aug 09 05:35:54 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:35:54.770413 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:54.753548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.30.137.195:35315)
0809 05:35:54.754118 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:54.754125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:54.754157 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:35:54.754227 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:54.754232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:54.754444 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:54.754580 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:54.756100 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.757103 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.758408 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:54.759084 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:54.759186 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:54.763948 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:54.763962 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:54.763978 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:54.764017 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:54.767653 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.768363 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.768373 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.768375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.768510 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:54.768862 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:54.768870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:54.768872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:54.769243 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:54.769479 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:54.769747 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 05:35:54.769964 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":277,"threads_started":1}
I20260809 05:35:54.771839 12285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:54.855940 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 12105
I20260809 05:35:54.880752 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 11559
I20260809 05:35:54.881996 11652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:35:54.988785 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 11559
I20260809 05:35:55.017127 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35515
--webserver_interface=127.30.137.254
--webserver_port=44299
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:35:55.361450 12316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:55.361768 12316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:55.361953 12316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:55.369303 12316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:35:55.369431 12316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:55.369521 12316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:55.369632 12316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:35:55.369701 12316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:35:55.379812 12316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=44299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:55.381199 12316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:55.382550 12316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:55.394793 12324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:55.394888 12321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:55.394888 12322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:55.395715 12316 server_base.cc:1034] running on GCE node
I20260809 05:35:55.396633 12316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:55.398109 12316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:55.399350 12316 hybrid_clock.cc:648] HybridClock initialized: now 1786253755399292 us; error 66 us; skew 500 ppm
I20260809 05:35:55.399719 12316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:55 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253755, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:55.405943 12316 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:35:55.407657 12316 webserver.cc:460] Webserver started at http://127.30.137.254:44299/ using document root <none> and password file <none>
I20260809 05:35:55.408408 12316 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:55.408579 12316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:55.414232 12316 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 05:35:55.417255 12334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:55.418797 12316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:35:55.418987 12316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "eef02030514a4316a3ef1d3c853ee263"
format_stamp: "Formatted at 2026-08-09 05:35:52 on dist-test-slave-3kk6"
server_key: "339c1efa389edd573cbb5e02e1f6334b"
server_key_iv: "33f1b3a68d182b5cc8135db3cc2f9b65"
server_key_version: "encryptionkey@0"
I20260809 05:35:55.420027 12316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:55.436630 12316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:55.440703 12316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:55.440991 12316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:55.460351 12316 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35515
I20260809 05:35:55.460376 12405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35515 every 8 connection(s)
I20260809 05:35:55.461947 12316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:35:55.466171 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 12316
I20260809 05:35:55.467259 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:44305
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:35515
--webserver_port=39701
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:35:55.469041 12406 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:35:55.472532 12406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Bootstrap starting.
I20260809 05:35:55.492547 12406 log.cc:824] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Log is configured to *not* fsync() on all Append() calls
I20260809 05:35:55.501757 12406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:35:55.502349 12406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Bootstrap complete.
I20260809 05:35:55.512102 12406 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:55.513049 12406 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef02030514a4316a3ef1d3c853ee263, State: Initialized, Role: FOLLOWER
I20260809 05:35:55.513741 12406 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:55.514003 12406 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:35:55.514134 12406 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:35:55.514338 12406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:35:55.518429 12406 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:55.519106 12406 leader_election.cc:302] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef02030514a4316a3ef1d3c853ee263; no voters:
I20260809 05:35:55.519809 12406 leader_election.cc:288] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 05:35:55.519963 12412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 05:35:55.521184 12412 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [term 2 LEADER]: Becoming Leader. State: Replica: eef02030514a4316a3ef1d3c853ee263, State: Running, Role: LEADER
I20260809 05:35:55.521867 12412 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } }
I20260809 05:35:55.522184 12406 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:35:55.524708 12414 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eef02030514a4316a3ef1d3c853ee263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } } }
I20260809 05:35:55.525163 12414 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:55.529857 12415 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef02030514a4316a3ef1d3c853ee263. Latest consensus state: current_term: 2 leader_uuid: "eef02030514a4316a3ef1d3c853ee263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef02030514a4316a3ef1d3c853ee263" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35515 } } }
I20260809 05:35:55.530874 12415 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263 [sys.catalog]: This master's current role is: LEADER
I20260809 05:35:55.534409 12422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:35:55.541220 12422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:35:55.543179 12422 catalog_manager.cc:1259] Loaded cluster ID: 868eacfb44ed40f5becfce85dfb80826
I20260809 05:35:55.543393 12422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:35:55.548739 12422 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:35:55.550762 12422 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eef02030514a4316a3ef1d3c853ee263: Loaded TSK: 0
I20260809 05:35:55.552855 12422 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 05:35:55.887373 12410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:55.887740 12410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:55.887909 12410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:55.896107 12410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:55.896265 12410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:55.896442 12410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:35:55.906816 12410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.193.keytab
--principal=oryx/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:44305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=39701
--tserver_master_addrs=127.30.137.254:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:55.908244 12410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:55.909628 12410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:55.924103 12448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:55.924129 12447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:55.924428 12451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:55.924909 12410 server_base.cc:1034] running on GCE node
I20260809 05:35:55.925855 12410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:55.927174 12410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:55.928445 12410 hybrid_clock.cc:648] HybridClock initialized: now 1786253755928392 us; error 64 us; skew 500 ppm
I20260809 05:35:55.928854 12410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:55 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253755, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:55.933794 12410 init.cc:377] Logged in from keytab as oryx/127.30.137.193@KRBTEST.COM (short username oryx)
I20260809 05:35:55.936349 12410 webserver.cc:460] Webserver started at http://127.30.137.193:39701/ using document root <none> and password file <none>
I20260809 05:35:55.937390 12410 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:55.937623 12410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:55.945319 12410 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 05:35:55.948920 12459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:55.950675 12410 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 05:35:55.950860 12410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "8b82d68f573d4d9f824a66ecf302aaec"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "467d11ae8b1bf53081c2bd2cf7d98c94"
server_key_iv: "769874e7c33a6adf545581f04f621330"
server_key_version: "encryptionkey@0"
I20260809 05:35:55.952237 12410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:55.975937 12410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:55.981783 12410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:55.982101 12410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:55.983400 12410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:55.985281 12410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:55.985396 12410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:55.985507 12410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:55.985579 12410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.021499 12410 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44305
I20260809 05:35:56.021600 12622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44305 every 8 connection(s)
I20260809 05:35:56.023108 12410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:35:56.028684 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 12410
I20260809 05:35:56.029350 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:34711
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:35515
--webserver_port=36877
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 05:35:56.044569 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.026414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48369 (local address 127.30.137.254:35515)
0809 05:35:56.027332 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.027348 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.027636 (+ 288us) server_negotiation.cc:407] Connection header received
0809 05:35:56.030020 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.030043 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.030504 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.030961 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.032542 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.034627 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.036393 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.037512 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.037724 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.043670 (+ 5946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:48369: BlockingRecv error: recv got EOF from 127.30.137.193:48369 (error 108)
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":342,"threads_started":1}
I20260809 05:35:56.044628 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.026534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:48369)
0809 05:35:56.027454 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.027501 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.029313 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.031058 (+ 1745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.031066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.031788 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.032327 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.032345 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.034857 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.034861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.036208 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.036251 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.037763 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.037769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.037965 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.038775 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.038799 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.043365 (+ 4566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":370,"threads_started":1}
W20260809 05:35:56.045400 12623 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:35515 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.137.254:35515: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:56.054782 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.046062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:57475)
0809 05:35:56.046262 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.046282 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.046380 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.046765 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.046768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.047087 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.047223 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.047230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.048700 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.048703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.049579 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.049589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.050622 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.050626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.050810 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.051437 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.051455 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.054512 (+ 3057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 05:35:56.055186 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.046205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:57475 (local address 127.30.137.254:35515)
0809 05:35:56.046404 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.046412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.046433 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:35:56.046492 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.046499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.046580 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.046736 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.047368 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.048562 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.049730 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.050482 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.050626 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.054861 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:57475: BlockingRecv error: recv got EOF from 127.30.137.193:57475 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:56.063954 12628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.055641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:59453)
0809 05:35:56.055863 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.055886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.056027 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.056391 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.056397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.056712 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.056930 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.056940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.058073 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.058079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.059088 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.059100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.059793 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.059799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.060013 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.060638 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.060656 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.063623 (+ 2967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260809 05:35:56.064100 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.055697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:59453 (local address 127.30.137.254:35515)
0809 05:35:56.055853 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.055857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.055972 (+ 115us) server_negotiation.cc:407] Connection header received
0809 05:35:56.056158 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.056163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.056246 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.056355 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.057070 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.057911 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.059238 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.059648 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.059782 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.063842 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:59453: BlockingRecv error: recv got EOF from 127.30.137.193:59453 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 05:35:56.064429 12623 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 05:35:56.386716 12630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:56.387079 12630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:56.387279 12630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:56.395182 12630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:56.395340 12630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:56.395533 12630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:35:56.405766 12630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.194.keytab
--principal=oryx/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:34711
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=36877
--tserver_master_addrs=127.30.137.254:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:56.407298 12630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:56.408614 12630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:56.422235 12638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:56.422315 12639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:56.422403 12644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:56.423497 12630 server_base.cc:1034] running on GCE node
I20260809 05:35:56.424175 12630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:56.425132 12630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:56.426370 12630 hybrid_clock.cc:648] HybridClock initialized: now 1786253756426336 us; error 48 us; skew 500 ppm
I20260809 05:35:56.426954 12630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:56.430249 12630 init.cc:377] Logged in from keytab as oryx/127.30.137.194@KRBTEST.COM (short username oryx)
I20260809 05:35:56.431782 12630 webserver.cc:460] Webserver started at http://127.30.137.194:36877/ using document root <none> and password file <none>
I20260809 05:35:56.432473 12630 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:56.432643 12630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:56.438241 12630 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:35:56.440913 12653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.442152 12630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:35:56.442329 12630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "4e4006e989654c0da81a9a746c860873"
format_stamp: "Formatted at 2026-08-09 05:35:53 on dist-test-slave-3kk6"
server_key: "0f0bde5c7f354ddb42308659f1a257e6"
server_key_iv: "cc78f7a15ab79f90d8519f41d94882e4"
server_key_version: "encryptionkey@0"
I20260809 05:35:56.443485 12630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:56.457216 12630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:56.461706 12630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:56.462024 12630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:56.463169 12630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:56.464540 12630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:56.464681 12630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.464813 12630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:56.464910 12630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.500058 12630 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:34711
I20260809 05:35:56.500159 12806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:34711 every 8 connection(s)
I20260809 05:35:56.501731 12630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:35:56.503934 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 12630
I20260809 05:35:56.504782 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:35315
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:35515
--webserver_port=40417
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 05:35:56.520056 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.504640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:58547 (local address 127.30.137.254:35515)
0809 05:35:56.504827 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.504832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.506261 (+ 1429us) server_negotiation.cc:407] Connection header received
0809 05:35:56.508228 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.508235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.508331 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.508448 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.510026 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.510824 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.513085 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.513876 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.514041 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.519708 (+ 5667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:58547: BlockingRecv error: recv got EOF from 127.30.137.194:58547 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 05:35:56.520509 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.505283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:58547)
0809 05:35:56.506079 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.506118 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.507938 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.508695 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.508703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.509253 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.509833 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.509852 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.510997 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.511001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.512892 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.512916 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.514130 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.514138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.514459 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.515349 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.515382 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.519393 (+ 4011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":334,"threads_started":1}
W20260809 05:35:56.521250 12807 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:35515 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.137.254:35515: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:56.530278 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.521946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:33167)
0809 05:35:56.522190 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.522212 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.522377 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.522721 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.522725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.523060 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.523268 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.523276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.524492 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.524497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.525461 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.525472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.526202 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.526208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.526391 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.527052 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.527069 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.529980 (+ 2911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 05:35:56.530448 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.522004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:33167 (local address 127.30.137.254:35515)
0809 05:35:56.522189 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.522194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.522291 (+ 97us) server_negotiation.cc:407] Connection header received
0809 05:35:56.522491 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.522499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.522562 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.522693 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.523405 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.524361 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.525610 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.526055 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.526186 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.530201 (+ 4015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:33167: BlockingRecv error: recv got EOF from 127.30.137.194:33167 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:56.540477 12809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.531398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:58395)
0809 05:35:56.531581 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.531600 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.531714 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.532096 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.532099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.532404 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.532554 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.532562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.534153 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.534157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.535188 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.535200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.536385 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.536392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.536561 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.537308 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.537327 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.540254 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 05:35:56.540756 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.531520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:58395 (local address 127.30.137.254:35515)
0809 05:35:56.531745 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.531753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.531774 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:35:56.531834 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.531842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.531940 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.532060 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.532704 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.533974 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.535385 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.536204 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.536387 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.540527 (+ 4140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:58395: BlockingRecv error: recv got EOF from 127.30.137.194:58395 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 05:35:56.540959 12807 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 05:35:56.858114 12810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:35:56.858412 12810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:35:56.858608 12810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:35:56.865890 12810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:35:56.866008 12810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:35:56.866127 12810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:35:56.876040 12810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/krb5kdc/oryx_127.30.137.195.keytab
--principal=oryx/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:35315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=40417
--tserver_master_addrs=127.30.137.254:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:35:56.877359 12810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:35:56.878628 12810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:35:56.892041 12821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:56.892015 12818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:35:56.892423 12817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:35:56.893023 12810 server_base.cc:1034] running on GCE node
I20260809 05:35:56.893620 12810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:35:56.894554 12810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:35:56.895795 12810 hybrid_clock.cc:648] HybridClock initialized: now 1786253756895744 us; error 71 us; skew 500 ppm
I20260809 05:35:56.896134 12810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:35:56.899605 12810 init.cc:377] Logged in from keytab as oryx/127.30.137.195@KRBTEST.COM (short username oryx)
I20260809 05:35:56.901291 12810 webserver.cc:460] Webserver started at http://127.30.137.195:40417/ using document root <none> and password file <none>
I20260809 05:35:56.901996 12810 fs_manager.cc:359] Metadata directory not provided
I20260809 05:35:56.902120 12810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:35:56.907991 12810 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 05:35:56.910773 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.912263 12810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:35:56.912449 12810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "06b827e03d35496c8d1778ae6c0661af"
format_stamp: "Formatted at 2026-08-09 05:35:54 on dist-test-slave-3kk6"
server_key: "cb93563bf62d3e423cc2096b2114e067"
server_key_iv: "478f4e4dea7eaec98717ffb31a6742f1"
server_key_version: "encryptionkey@0"
I20260809 05:35:56.913440 12810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:35:56.932806 12810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:35:56.937564 12810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:35:56.937865 12810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:35:56.938964 12810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:35:56.940351 12810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:35:56.940487 12810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.940629 12810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:35:56.940727 12810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:35:56.976461 12810 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:35315
I20260809 05:35:56.976540 12988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:35315 every 8 connection(s)
I20260809 05:35:56.977959 12810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:35:56.987040 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 12810
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:56 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 05:35:56.995767 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.980855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:33977 (local address 127.30.137.254:35515)
0809 05:35:56.981048 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.981053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.982258 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 05:35:56.984138 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.984143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.984212 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.984304 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.985838 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.986622 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.988789 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.989497 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.989632 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.995517 (+ 5885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:33977: BlockingRecv error: recv got EOF from 127.30.137.195:33977 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:35:56.996842 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.981352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:33977)
0809 05:35:56.982085 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.982124 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.983873 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.984518 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.984526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.985098 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.985682 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.985700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.986803 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.986807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.988569 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.988588 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.989727 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.989734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:56.989978 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.990756 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:56.990778 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:56.995341 (+ 4563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":300,"threads_started":1}
W20260809 05:35:56.997615 12991 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:35515 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.137.254:35515: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
I20260809 05:35:57.005492 12999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.988105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52784 (local address 127.30.137.254:35515)
0809 05:35:56.988959 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.988965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.988981 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:35:56.989032 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.989036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.989109 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.989241 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:56.990732 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.991513 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.993183 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:56.993794 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:56.993893 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:56.997599 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:35:56.997608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:35:56.997624 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:35:56.997662 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:35:57.001555 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:57.002395 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:57.002408 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:57.002420 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:57.002727 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:35:57.003196 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:35:57.003199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:35:57.003201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:35:57.003922 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:35:57.004268 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 05:35:57.004758 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 05:35:57.005182 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":611,"threads_started":1}
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:57.007843 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:56.998238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:43613)
0809 05:35:56.998476 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:56.998494 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:56.998603 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:56.999283 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:56.999291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:56.999727 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:56.999964 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:56.999978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.001069 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.001079 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.002270 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.002310 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.003398 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.003405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.003596 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.004246 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:57.004264 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:57.007575 (+ 3311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 05:35:57.007975 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:56.998819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:43613 (local address 127.30.137.254:35515)
0809 05:35:56.998979 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:56.998983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:56.998998 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:35:56.999053 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:56.999060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:56.999121 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:56.999211 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:57.000130 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.000914 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.002461 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.003196 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.003310 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.007745 (+ 4435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:43613: BlockingRecv error: recv got EOF from 127.30.137.195:43613 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:57.016580 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:57.008806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:35705)
0809 05:35:57.009010 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:57.009026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:57.009106 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:57.009500 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:57.009505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:57.009892 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:57.010052 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.010062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.011208 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.011211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.012175 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.012187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.012813 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.012817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.012960 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.013497 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:57.013513 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:57.016243 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 05:35:57.016671 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:57.008863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:35705 (local address 127.30.137.254:35515)
0809 05:35:57.009251 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:57.009254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:57.009266 (+ 12us) server_negotiation.cc:407] Connection header received
0809 05:35:57.009302 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:57.009306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:57.009357 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:57.009434 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:57.010206 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.011084 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.012302 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.012691 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.012785 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.016432 (+ 3647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:35705: BlockingRecv error: recv got EOF from 127.30.137.195:35705 (error 108)
Metrics: {"server-negotiator.queue_time_us":297}
W20260809 05:35:57.016944 12991 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:57.074170 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:57.065101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:45715)
0809 05:35:57.065713 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:57.065729 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:57.065909 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:57.066270 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:57.066275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:57.066903 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:57.067108 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.067117 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.068210 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.068215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.069177 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.069188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.069993 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.069999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.070172 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.070918 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:57.070935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:57.073821 (+ 2886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":350,"threads_started":1}
W20260809 05:35:57.074304 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:57.065203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:45715 (local address 127.30.137.254:35515)
0809 05:35:57.065546 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:57.065552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:57.065831 (+ 279us) server_negotiation.cc:407] Connection header received
0809 05:35:57.066059 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:57.066063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:57.066127 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:57.066232 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:57.067292 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.068041 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.069326 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.069792 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.069930 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.074056 (+ 4126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:45715: BlockingRecv error: recv got EOF from 127.30.137.193:45715 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:57 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:57.551005 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:57.541723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:56025)
0809 05:35:57.542323 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:57.542340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:57.542471 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:57.542886 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:57.542889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:57.543379 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:57.543581 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.543611 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.544907 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.544911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.545969 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:57.545982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.546798 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.546804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:57.547004 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.547656 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:57.547673 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:57.550764 (+ 3091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":342,"threads_started":1}
W20260809 05:35:57.551205 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:57.541868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:56025 (local address 127.30.137.254:35515)
0809 05:35:57.542053 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:57.542057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:57.542428 (+ 371us) server_negotiation.cc:407] Connection header received
0809 05:35:57.542577 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:57.542581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:57.542728 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:57.542828 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:57.543781 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.544728 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.546117 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:57.546578 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:57.546746 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:57.550945 (+ 4199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:56025: BlockingRecv error: recv got EOF from 127.30.137.194:56025 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 05:35:58 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:58 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:58.027282 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:58.017499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:42809)
0809 05:35:58.018107 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:58.018124 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:58.018253 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:58.018562 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:58.018565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:58.019065 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:58.019390 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:58.019404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.020388 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.020392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:58.021972 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:58.021990 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.022774 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.022783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:58.022970 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:58.023705 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:58.023751 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:58.026925 (+ 3174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":353,"threads_started":1}
W20260809 05:35:58.027392 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:58.017588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:42809 (local address 127.30.137.254:35515)
0809 05:35:58.017864 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:58.017882 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:58.018209 (+ 327us) server_negotiation.cc:407] Connection header received
0809 05:35:58.018356 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:58.018360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:58.018432 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:58.018513 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:58.019541 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.020251 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.022113 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.022522 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.022617 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:58.027147 (+ 4530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:42809: BlockingRecv error: recv got EOF from 127.30.137.195:42809 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 09 05:35:58 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:58 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:58.084507 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:58.075363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:59177)
0809 05:35:58.075966 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:58.075983 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:58.076135 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:58.076613 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:58.076617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:58.077200 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:58.077458 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:58.077474 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.078547 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.078550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:58.079485 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:58.079496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.080202 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.080206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:58.080324 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:58.081035 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:58.081054 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:58.084128 (+ 3074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":346,"threads_started":1}
W20260809 05:35:58.084786 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:58.075555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:59177 (local address 127.30.137.254:35515)
0809 05:35:58.075747 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:58.075751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:58.076075 (+ 324us) server_negotiation.cc:407] Connection header received
0809 05:35:58.076279 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:58.076305 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:58.076420 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:58.076547 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:58.077670 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.078425 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.079622 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.080064 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.080163 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:58.084359 (+ 4196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:59177: BlockingRecv error: recv got EOF from 127.30.137.193:59177 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 05:35:58 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:58 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:58.561767 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:58.552073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:55877)
0809 05:35:58.552743 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:58.552760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:58.552904 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:58.553242 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:58.553246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:58.553655 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:58.553868 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:58.553877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.554985 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.554990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:58.556169 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:58.556183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.557175 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.557185 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:58.557371 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:58.558082 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:58.558100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:58.561375 (+ 3275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":405,"threads_started":1}
W20260809 05:35:58.561831 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:58.552204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:55877 (local address 127.30.137.254:35515)
0809 05:35:58.552425 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:58.552430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:58.552879 (+ 449us) server_negotiation.cc:407] Connection header received
0809 05:35:58.553019 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:58.553023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:58.553095 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:58.553200 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:58.554017 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.554854 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.556311 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:58.556877 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:58.557015 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:58.561599 (+ 4584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:55877: BlockingRecv error: recv got EOF from 127.30.137.194:55877 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 09 05:35:59 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:59 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:59.037392 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:59.028291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:39771)
0809 05:35:59.028944 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:59.028960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:59.029136 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:59.029511 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:59.029516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:59.029967 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:59.030151 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:59.030161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.031292 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.031298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:59.032241 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:59.032253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.032990 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.032996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:59.033181 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:59.033853 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:59.033871 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:59.037039 (+ 3168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":387,"threads_started":1}
W20260809 05:35:59.037623 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:59.028488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:39771 (local address 127.30.137.254:35515)
0809 05:35:59.028713 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:59.028722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:59.029052 (+ 330us) server_negotiation.cc:407] Connection header received
0809 05:35:59.029289 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:59.029300 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:59.029363 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:59.029470 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:59.030303 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.031151 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.032389 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.032829 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.032973 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:59.037289 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:39771: BlockingRecv error: recv got EOF from 127.30.137.195:39771 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 05:35:59 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:59 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:59.094173 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:59.085740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:42211)
0809 05:35:59.086493 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:59.086515 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:59.086753 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:59.087139 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:59.087142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:59.087504 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:59.087675 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:59.087683 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.088745 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.088749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:59.089641 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:59.089651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.090314 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.090318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:59.090446 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:59.091065 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:59.091082 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:59.093912 (+ 2830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":438,"threads_started":1}
W20260809 05:35:59.094391 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:59.085743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:42211 (local address 127.30.137.254:35515)
0809 05:35:59.086002 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:59.086012 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:59.086633 (+ 621us) server_negotiation.cc:407] Connection header received
0809 05:35:59.086924 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:59.086932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:59.087009 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:59.087091 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:59.087835 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.088588 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.089782 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.090167 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.090301 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:59.094108 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:42211: BlockingRecv error: recv got EOF from 127.30.137.193:42211 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 05:35:59 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:35:59 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:35:59.571730 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:35:59.563020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:56749)
0809 05:35:59.563635 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 05:35:59.563653 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:35:59.563783 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:35:59.564104 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:35:59.564108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:35:59.564483 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:35:59.564665 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:59.564673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.565914 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.565918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:59.566910 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:35:59.566924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.567682 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.567688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:35:59.567881 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:59.568508 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:35:59.568526 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:35:59.571447 (+ 2921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":332,"threads_started":1}
W20260809 05:35:59.571928 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:35:59.563099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:56749 (local address 127.30.137.254:35515)
0809 05:35:59.563285 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 05:35:59.563289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:35:59.563742 (+ 453us) server_negotiation.cc:407] Connection header received
0809 05:35:59.563893 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:35:59.563897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:35:59.563960 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:35:59.564052 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:35:59.564868 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.565743 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.567060 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:35:59.567524 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:35:59.567665 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:35:59.571659 (+ 3994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:56749: BlockingRecv error: recv got EOF from 127.30.137.194:56749 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 05:36:00 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:00 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:00.048002 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:00.038545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:54331)
0809 05:36:00.039234 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:00.039252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:00.039428 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:00.039811 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:00.039818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:00.040225 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:00.040410 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:00.040420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.041487 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.041495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:00.042429 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:00.042441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.043262 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.043268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:00.043458 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:00.044181 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:00.044225 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:00.047614 (+ 3389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":377,"threads_started":1}
W20260809 05:36:00.048084 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:00.038777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:54331 (local address 127.30.137.254:35515)
0809 05:36:00.039026 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:00.039034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:00.039349 (+ 315us) server_negotiation.cc:407] Connection header received
0809 05:36:00.039564 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:00.039570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:00.039646 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:00.039765 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:00.040574 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.041341 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.042577 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.043095 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.043255 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:00.047843 (+ 4588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:54331: BlockingRecv error: recv got EOF from 127.30.137.195:54331 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 05:36:00 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:00 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:00.106602 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:00.095433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:33273)
0809 05:36:00.096221 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:00.096240 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:00.096426 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:00.097024 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:00.097031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:00.097634 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:00.097942 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:00.097953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.099079 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.099085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:00.100094 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:00.100105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.100887 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.100892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:00.101086 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:00.101906 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:00.101935 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:00.106156 (+ 4221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":476,"threads_started":1}
W20260809 05:36:00.107442 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:00.095620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:33273 (local address 127.30.137.254:35515)
0809 05:36:00.095945 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:00.095952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:00.096348 (+ 396us) server_negotiation.cc:407] Connection header received
0809 05:36:00.096594 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:00.096600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:00.096682 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:00.096773 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:00.098100 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.098941 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.100259 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.100721 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.100862 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:00.107221 (+ 6359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:33273: BlockingRecv error: recv got EOF from 127.30.137.193:33273 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Aug 09 05:36:00 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:00 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:00.587543 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:00.572845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:47829)
0809 05:36:00.575778 (+ 2933us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:00.575802 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:00.576011 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:00.576686 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:00.576693 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:00.577239 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:00.577498 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:00.577510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.579080 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.579085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:00.580127 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:00.580140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.581027 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.581036 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:00.581237 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:00.582056 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:00.582083 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:00.586533 (+ 4450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2781,"thread_start_us":2300,"threads_started":1}
W20260809 05:36:00.587891 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:00.574889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:47829 (local address 127.30.137.254:35515)
0809 05:36:00.575071 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:00.575077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:00.575885 (+ 808us) server_negotiation.cc:407] Connection header received
0809 05:36:00.576184 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:00.576191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:00.576302 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:00.576382 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:00.577674 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.578918 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.580297 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:00.580844 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:00.580998 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:00.587499 (+ 6501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:47829: BlockingRecv error: recv got EOF from 127.30.137.194:47829 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 05:36:01 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:01 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:01.058549 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:01.049180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:39737)
0809 05:36:01.049794 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:01.049813 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:01.049948 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:01.050281 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:01.050285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:01.050740 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:01.051011 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:01.051025 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.052185 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.052189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:01.053185 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:01.053197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.053929 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.053933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:01.054060 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:01.054765 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:01.054784 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:01.058313 (+ 3529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":369,"threads_started":1}
W20260809 05:36:01.058908 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:01.049251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:39737 (local address 127.30.137.254:35515)
0809 05:36:01.049450 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:01.049460 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:01.049900 (+ 440us) server_negotiation.cc:407] Connection header received
0809 05:36:01.050062 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:01.050066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:01.050135 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:01.050231 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:01.051183 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.052047 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.053334 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.053785 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.053886 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:01.058566 (+ 4680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:39737: BlockingRecv error: recv got EOF from 127.30.137.195:39737 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 05:36:01 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:01 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:01.116993 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:01.107918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:48455)
0809 05:36:01.108529 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:01.108546 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:01.108722 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:01.109078 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:01.109083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:01.109561 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:01.109735 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:01.109744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.110940 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.110947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:01.111906 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:01.111917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.112674 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.112680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:01.112878 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:01.113568 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:01.113586 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:01.116671 (+ 3085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":303,"threads_started":1}
W20260809 05:36:01.117170 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:01.107982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48455 (local address 127.30.137.254:35515)
0809 05:36:01.108267 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:01.108274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:01.108661 (+ 387us) server_negotiation.cc:407] Connection header received
0809 05:36:01.108864 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:01.108869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:01.108937 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:01.109044 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:01.109930 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.110798 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.112063 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.112504 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.112643 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:01.116915 (+ 4272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:48455: BlockingRecv error: recv got EOF from 127.30.137.193:48455 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 05:36:01 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:01 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:01.599238 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:01.590449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:33121)
0809 05:36:01.591071 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:01.591089 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:01.591216 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:01.591609 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:01.591612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:01.592125 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:01.592292 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:01.592301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.593376 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.593380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:01.594329 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:01.594342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.595061 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.595065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:01.595188 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:01.595900 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:01.595917 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:01.598968 (+ 3051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":335,"threads_started":1}
W20260809 05:36:01.599609 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:01.590655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:33121 (local address 127.30.137.254:35515)
0809 05:36:01.590904 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:01.590911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:01.591179 (+ 268us) server_negotiation.cc:407] Connection header received
0809 05:36:01.591378 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:01.591384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:01.591451 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:01.591555 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:01.592457 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.593229 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.594457 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:01.594930 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:01.595027 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:01.599226 (+ 4199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:33121: BlockingRecv error: recv got EOF from 127.30.137.194:33121 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 05:36:02 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:02 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:02.068660 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:02.059625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:43655)
0809 05:36:02.060221 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:02.060239 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:02.060367 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:02.060679 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:02.060682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:02.061125 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:02.061291 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:02.061299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.062400 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.062405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:02.063461 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:02.063475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.064217 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.064221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:02.064357 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:02.065052 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:02.065072 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:02.068280 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":292,"threads_started":1}
W20260809 05:36:02.068832 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:02.059700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:43655 (local address 127.30.137.254:35515)
0809 05:36:02.059908 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:02.059918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:02.060326 (+ 408us) server_negotiation.cc:407] Connection header received
0809 05:36:02.060472 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:02.060476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:02.060543 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:02.060619 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:02.061485 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.062251 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.063633 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.064073 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.064186 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:02.068562 (+ 4376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:43655: BlockingRecv error: recv got EOF from 127.30.137.195:43655 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 05:36:02 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:02 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:02.127856 13027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:02.118182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:48361)
0809 05:36:02.118817 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:02.118836 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:02.118972 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:02.119339 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:02.119343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:02.119834 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:02.120003 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:02.120012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.121068 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.121072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:02.122075 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:02.122091 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.123236 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.123244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:02.123471 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:02.124330 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:02.124350 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:02.127616 (+ 3266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":320,"threads_started":1}
W20260809 05:36:02.128168 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:02.118377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48361 (local address 127.30.137.254:35515)
0809 05:36:02.118683 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:02.118690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:02.118935 (+ 245us) server_negotiation.cc:407] Connection header received
0809 05:36:02.119084 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:02.119089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:02.119156 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:02.119261 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:02.120146 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.120918 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.122270 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.122991 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.123156 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:02.127861 (+ 4705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:48361: BlockingRecv error: recv got EOF from 127.30.137.193:48361 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 09 05:36:02 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:02 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:02.610085 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:02.600642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:53805)
0809 05:36:02.601306 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:02.601324 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:02.601458 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:02.601885 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:02.601888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:02.602355 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:02.602514 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:02.602523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.603805 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.603810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:02.605048 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:02.605064 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.605929 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.605934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:02.606066 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:02.606824 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:02.606842 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:02.609846 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":373,"threads_started":1}
W20260809 05:36:02.610483 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:02.600792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:53805 (local address 127.30.137.254:35515)
0809 05:36:02.601004 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:02.601009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:02.601446 (+ 437us) server_negotiation.cc:407] Connection header received
0809 05:36:02.601569 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:02.601573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:02.601642 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:02.601753 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:02.602695 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.603616 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.605252 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:02.605772 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:02.605879 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:02.610134 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:53805: BlockingRecv error: recv got EOF from 127.30.137.194:53805 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 05:36:03 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:03 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:03.079476 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:03.069976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:34767)
0809 05:36:03.070552 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:03.070570 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:03.070794 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:03.071192 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:03.071195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:03.071793 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:03.071995 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:03.072009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.073608 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.073612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:03.074555 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:03.074568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.075414 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.075419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:03.075548 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:03.076202 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:03.076220 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:03.079239 (+ 3019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":323,"threads_started":1}
W20260809 05:36:03.079914 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:03.070232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:34767 (local address 127.30.137.254:35515)
0809 05:36:03.070523 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:03.070531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:03.070693 (+ 162us) server_negotiation.cc:407] Connection header received
0809 05:36:03.070918 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:03.070923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:03.071028 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:03.071152 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:03.072178 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.073415 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.074785 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.075257 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.075385 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:03.079489 (+ 4104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:34767: BlockingRecv error: recv got EOF from 127.30.137.195:34767 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 09 05:36:03 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:03 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:03.138381 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:03.129034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:52485)
0809 05:36:03.129651 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:03.129668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:03.129860 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:03.130249 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:03.130254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:03.130804 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:03.130986 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:03.130995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.132031 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.132041 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:03.132967 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:03.132977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.133705 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.133711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:03.133874 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:03.134591 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:03.134617 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:03.138106 (+ 3489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":296,"threads_started":1}
W20260809 05:36:03.138631 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:03.129096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:52485 (local address 127.30.137.254:35515)
0809 05:36:03.129344 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:03.129352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:03.129770 (+ 418us) server_negotiation.cc:407] Connection header received
0809 05:36:03.130024 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:03.130030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:03.130104 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:03.130210 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:03.131131 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.131881 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.133106 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.133528 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.133666 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:03.138384 (+ 4718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:52485: BlockingRecv error: recv got EOF from 127.30.137.193:52485 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 05:36:03 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:03 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:03.620939 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:03.611434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:38587)
0809 05:36:03.612196 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:03.612215 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:03.612402 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:03.612844 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:03.612850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:03.613548 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:03.613756 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:03.613767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.614880 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.614885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:03.615859 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:03.615873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.616565 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.616570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:03.616694 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:03.617386 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:03.617405 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:03.620550 (+ 3145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":390,"threads_started":1}
W20260809 05:36:03.621107 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:03.611549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:38587 (local address 127.30.137.254:35515)
0809 05:36:03.611859 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:03.611869 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:03.612331 (+ 462us) server_negotiation.cc:407] Connection header received
0809 05:36:03.612552 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:03.612556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:03.612652 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:03.612759 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:03.613903 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.614714 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.615993 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:03.616421 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:03.616514 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:03.620814 (+ 4300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:38587: BlockingRecv error: recv got EOF from 127.30.137.194:38587 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 09 05:36:04 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:04 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:04.090101 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:04.080489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:39541)
0809 05:36:04.081120 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:04.081137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:04.081263 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:04.081597 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:04.081600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:04.082037 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:04.082222 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:04.082231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.083344 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.083348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:04.084286 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:04.084310 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.085089 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.085094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:04.085221 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:04.085908 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:04.085925 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:04.089696 (+ 3771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":300,"threads_started":1}
W20260809 05:36:04.090144 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:04.080659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:39541 (local address 127.30.137.254:35515)
0809 05:36:04.080890 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:04.080895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:04.081222 (+ 327us) server_negotiation.cc:407] Connection header received
0809 05:36:04.081377 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:04.081382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:04.081447 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:04.081538 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:04.082376 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.083195 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.084466 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.084931 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.085047 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:04.089975 (+ 4928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:39541: BlockingRecv error: recv got EOF from 127.30.137.195:39541 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 09 05:36:04 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:04 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:04.148805 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:04.139487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:49997)
0809 05:36:04.140093 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:04.140122 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:04.140288 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:04.140782 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:04.140790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:04.141439 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:04.141612 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:04.141623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.142769 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.142774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:04.143758 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:04.143772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.144531 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.144536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:04.144649 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:04.145296 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:04.145315 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:04.148505 (+ 3190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":310,"threads_started":1}
W20260809 05:36:04.149049 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:04.139658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:49997 (local address 127.30.137.254:35515)
0809 05:36:04.139880 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:04.139888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:04.140223 (+ 335us) server_negotiation.cc:407] Connection header received
0809 05:36:04.140436 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:04.140442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:04.140592 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:04.140680 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:04.141786 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.142542 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.143913 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.144363 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.144469 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:04.148770 (+ 4301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:49997: BlockingRecv error: recv got EOF from 127.30.137.193:49997 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 05:36:04 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:04 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:04.631771 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:04.622420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:36449)
0809 05:36:04.623032 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:04.623049 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:04.623227 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:04.623601 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:04.623605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:04.624099 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:04.624317 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:04.624336 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.625698 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.625702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:04.626640 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:04.626722 (+ 82us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.627557 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.627563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:04.627728 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:04.628382 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:04.628400 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:04.631378 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":369,"threads_started":1}
W20260809 05:36:04.631877 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:04.622536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:36449 (local address 127.30.137.254:35515)
0809 05:36:04.622830 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:04.622836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:04.623159 (+ 323us) server_negotiation.cc:407] Connection header received
0809 05:36:04.623366 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:04.623370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:04.623436 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:04.623540 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:04.624461 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.625515 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.626853 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:04.627362 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:04.627504 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:04.631582 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:36449: BlockingRecv error: recv got EOF from 127.30.137.194:36449 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 05:36:05 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:05 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:05.100472 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:05.091304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:51653)
0809 05:36:05.091850 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:05.091869 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:05.091992 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:05.092336 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:05.092342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:05.092780 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:05.092959 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:05.092967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.094183 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.094187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:05.095394 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:05.095408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.096221 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.096228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:05.096357 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:05.097048 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:05.097067 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:05.100210 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":288,"threads_started":1}
W20260809 05:36:05.100816 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:05.091366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:51653 (local address 127.30.137.254:35515)
0809 05:36:05.091611 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:05.091620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:05.091955 (+ 335us) server_negotiation.cc:407] Connection header received
0809 05:36:05.092098 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:05.092103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:05.092171 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:05.092280 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:05.093153 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.094016 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.095536 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.095994 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.096098 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:05.100454 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:51653: BlockingRecv error: recv got EOF from 127.30.137.195:51653 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 05:36:05 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:05 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:05.158973 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:05.149853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:46527)
0809 05:36:05.150450 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:05.150467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:05.150643 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:05.151116 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:05.151121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:05.151569 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:05.151753 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:05.151761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.152846 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.152852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:05.153826 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:05.153845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.154576 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.154581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:05.154792 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:05.155450 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:05.155494 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:05.158551 (+ 3057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":325,"threads_started":1}
W20260809 05:36:05.159067 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:05.149938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:46527 (local address 127.30.137.254:35515)
0809 05:36:05.150191 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:05.150200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:05.150565 (+ 365us) server_negotiation.cc:407] Connection header received
0809 05:36:05.150862 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:05.150867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:05.150950 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:05.151063 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:05.151912 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.152694 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.153988 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.154413 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.154577 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:05.158821 (+ 4244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:46527: BlockingRecv error: recv got EOF from 127.30.137.193:46527 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 05:36:05 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:05 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:05.646185 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:05.632754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:34251)
0809 05:36:05.633319 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:05.633337 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:05.633514 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:05.633889 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:05.633894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:05.634350 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:05.634609 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:05.634622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.635902 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.635909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:05.636886 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:05.636899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.637653 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.637670 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:05.637861 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:05.642001 (+ 4140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:05.642021 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:05.645771 (+ 3750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":310,"threads_started":1}
W20260809 05:36:05.646337 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:05.632793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:34251 (local address 127.30.137.254:35515)
0809 05:36:05.633055 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:05.633063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:05.633431 (+ 368us) server_negotiation.cc:407] Connection header received
0809 05:36:05.633662 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:05.633668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:05.633730 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:05.633836 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:05.634891 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.635767 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.637042 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:05.637477 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:05.637616 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:05.646054 (+ 8438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:34251: BlockingRecv error: recv got EOF from 127.30.137.194:34251 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 05:36:06 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:06 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:06.110890 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:06.101710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:49187)
0809 05:36:06.102323 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:06.102351 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:06.102520 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:06.102921 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:06.102925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:06.103344 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:06.103522 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:06.103530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.104588 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.104593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:06.105604 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:06.105618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.106372 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.106377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:06.106508 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:06.107288 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:06.107307 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:06.110479 (+ 3172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":312,"threads_started":1}
W20260809 05:36:06.111048 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:06.101745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:49187 (local address 127.30.137.254:35515)
0809 05:36:06.101955 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:06.101964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:06.102477 (+ 513us) server_negotiation.cc:407] Connection header received
0809 05:36:06.102631 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:06.102634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:06.102764 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:06.102856 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:06.103703 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.104449 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.105765 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.106202 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.106315 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:06.110833 (+ 4518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:49187: BlockingRecv error: recv got EOF from 127.30.137.195:49187 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 05:36:06 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:06 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:06.168702 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:06.160166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:48691)
0809 05:36:06.160790 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:06.160815 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:06.160963 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:06.161408 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:06.161412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:06.161808 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:06.161975 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:06.161983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.163054 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.163059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:06.164016 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:06.164027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.164716 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.164720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:06.164854 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:06.165515 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:06.165534 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:06.168462 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":340,"threads_started":1}
W20260809 05:36:06.169155 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:06.160297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48691 (local address 127.30.137.254:35515)
0809 05:36:06.160496 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:06.160501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:06.160892 (+ 391us) server_negotiation.cc:407] Connection header received
0809 05:36:06.161104 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:06.161110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:06.161218 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:06.161361 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:06.162111 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.162918 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.164141 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.164573 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.164668 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:06.168691 (+ 4023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:48691: BlockingRecv error: recv got EOF from 127.30.137.193:48691 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 05:36:06 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:06 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:06.656503 13045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:06.647414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:42213)
0809 05:36:06.648021 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:06.648042 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:06.648249 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:06.648646 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:06.648652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:06.649116 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:06.649282 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:06.649292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.650359 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.650365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:06.651410 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:06.651423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.652215 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.652221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:06.652398 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:06.653170 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:06.653189 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:06.656263 (+ 3074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":318,"threads_started":1}
W20260809 05:36:06.656872 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:06.647499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:42213 (local address 127.30.137.254:35515)
0809 05:36:06.647709 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:06.647718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:06.648142 (+ 424us) server_negotiation.cc:407] Connection header received
0809 05:36:06.648402 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:06.648410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:06.648497 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:06.648607 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:06.649443 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.650205 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.651564 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:06.652030 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:06.652171 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:06.656514 (+ 4343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:42213: BlockingRecv error: recv got EOF from 127.30.137.194:42213 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 05:36:07 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:07 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:07.121815 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:07.111896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:45961)
0809 05:36:07.112556 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:07.112574 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:07.112751 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:07.113140 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:07.113145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:07.113611 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:07.113794 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:07.113803 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.115077 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.115084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:07.116150 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:07.116163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.117042 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.117048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:07.117253 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:07.118008 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:07.118026 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:07.121585 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":399,"threads_started":1}
W20260809 05:36:07.122077 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:07.112096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:45961 (local address 127.30.137.254:35515)
0809 05:36:07.112414 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:07.112421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:07.112673 (+ 252us) server_negotiation.cc:407] Connection header received
0809 05:36:07.112904 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:07.112910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:07.112986 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:07.113104 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:07.113968 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.114911 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.116317 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.116856 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.117008 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:07.121764 (+ 4756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:45961: BlockingRecv error: recv got EOF from 127.30.137.195:45961 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 09 05:36:07 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:07 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:07.178407 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:07.169713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:36925)
0809 05:36:07.170308 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:07.170329 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:07.170496 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:07.170919 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:07.170926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:07.171414 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:07.171593 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:07.171602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.172634 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.172641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:07.173551 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:07.173562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.174321 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.174327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:07.174515 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:07.175157 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:07.175174 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:07.178029 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":331,"threads_started":1}
W20260809 05:36:07.178486 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:07.169778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:36925 (local address 127.30.137.254:35515)
0809 05:36:07.170015 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:07.170023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:07.170427 (+ 404us) server_negotiation.cc:407] Connection header received
0809 05:36:07.170630 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:07.170635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:07.170761 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:07.170874 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:07.171758 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.172496 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.173695 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.174142 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.174276 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:07.178236 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:36925: BlockingRecv error: recv got EOF from 127.30.137.193:36925 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 05:36:07 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:07 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:07.666532 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:07.657508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:45931)
0809 05:36:07.658089 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:07.658106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:07.658227 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:07.658522 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:07.658525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:07.658990 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:07.659173 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:07.659182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.660704 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.660711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:07.661701 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:07.661716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.662477 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.662482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:07.662617 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:07.663393 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:07.663411 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:07.666280 (+ 2869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":307,"threads_started":1}
W20260809 05:36:07.666781 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:07.657560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:45931 (local address 127.30.137.254:35515)
0809 05:36:07.657812 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:07.657821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:07.658194 (+ 373us) server_negotiation.cc:407] Connection header received
0809 05:36:07.658324 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:07.658328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:07.658393 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:07.658475 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:07.659322 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.660505 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.661885 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:07.662319 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:07.662419 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:07.666511 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:45931: BlockingRecv error: recv got EOF from 127.30.137.194:45931 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 05:36:08 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:08 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:08.132043 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:08.122971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:49069)
0809 05:36:08.123561 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:08.123578 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:08.123750 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:08.124139 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:08.124145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:08.124612 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:08.124798 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:08.124807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.125873 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.125881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:08.126962 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:08.126975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.127825 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.127830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:08.128006 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:08.128608 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:08.128625 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:08.131679 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":339,"threads_started":1}
W20260809 05:36:08.132156 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:08.123060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:49069 (local address 127.30.137.254:35515)
0809 05:36:08.123316 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:08.123325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:08.123675 (+ 350us) server_negotiation.cc:407] Connection header received
0809 05:36:08.123892 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:08.123898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:08.123977 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:08.124084 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:08.124950 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.125728 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.127104 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.127668 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.127816 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:08.131911 (+ 4095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:49069: BlockingRecv error: recv got EOF from 127.30.137.195:49069 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 05:36:08 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:08 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:08.188267 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:08.179450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:37979)
0809 05:36:08.180053 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:08.180071 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:08.180211 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:08.180536 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:08.180539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:08.180977 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:08.181139 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:08.181147 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.182152 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.182156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:08.183216 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:08.183229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.184003 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.184008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:08.184144 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:08.184883 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:08.184902 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:08.187925 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":375,"threads_started":1}
W20260809 05:36:08.188505 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:08.179538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:37979 (local address 127.30.137.254:35515)
0809 05:36:08.179707 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:08.179711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:08.180144 (+ 433us) server_negotiation.cc:407] Connection header received
0809 05:36:08.180317 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:08.180321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:08.180398 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:08.180489 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:08.181283 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.182017 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.183366 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.183867 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.183982 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:08.188218 (+ 4236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:37979: BlockingRecv error: recv got EOF from 127.30.137.193:37979 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 05:36:08 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:08 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:08.676429 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:08.667760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:43083)
0809 05:36:08.668291 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:08.668319 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:08.668507 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:08.668870 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:08.668875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:08.669472 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:08.669629 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:08.669638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.670724 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.670730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:08.671673 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:08.671684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.672422 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.672427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:08.672614 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:08.673243 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:08.673260 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:08.676069 (+ 2809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":330,"threads_started":1}
W20260809 05:36:08.676553 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:08.667826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:43083 (local address 127.30.137.254:35515)
0809 05:36:08.668089 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:08.668099 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:08.668435 (+ 336us) server_negotiation.cc:407] Connection header received
0809 05:36:08.668649 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:08.668654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:08.668716 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:08.668833 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:08.669786 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.670539 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.671831 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:08.672269 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:08.672410 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:08.676311 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:43083: BlockingRecv error: recv got EOF from 127.30.137.194:43083 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 05:36:09 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:09 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:09.142877 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:09.133300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:44043)
0809 05:36:09.133829 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:09.133847 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:09.133988 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:09.134480 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:09.134483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:09.135055 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:09.135267 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:09.135277 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.136902 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.136907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:09.137866 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:09.137879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.138602 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.138606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:09.138974 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:09.139628 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:09.139646 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:09.142563 (+ 2917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":318,"threads_started":1}
W20260809 05:36:09.143043 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:09.133403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:44043 (local address 127.30.137.254:35515)
0809 05:36:09.133666 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:09.133673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:09.133942 (+ 269us) server_negotiation.cc:407] Connection header received
0809 05:36:09.134178 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:09.134188 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:09.134305 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:09.134424 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:09.135441 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.136719 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.138024 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.138455 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.138552 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:09.142801 (+ 4249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:44043: BlockingRecv error: recv got EOF from 127.30.137.195:44043 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 05:36:09 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:09 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:09.198544 13054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:09.189594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:54657)
0809 05:36:09.190195 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:09.190213 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:09.190331 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:09.190621 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:09.190624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:09.191051 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:09.191218 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:09.191227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.192243 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.192246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:09.193188 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:09.193198 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.193864 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.193868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:09.193981 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:09.194722 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:09.194741 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:09.198156 (+ 3415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":327,"threads_started":1}
W20260809 05:36:09.198634 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:09.189648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:54657 (local address 127.30.137.254:35515)
0809 05:36:09.189850 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:09.189854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:09.190291 (+ 437us) server_negotiation.cc:407] Connection header received
0809 05:36:09.190435 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:09.190439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:09.190523 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:09.190607 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:09.191343 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.192120 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.193310 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.193703 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.193792 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:09.198393 (+ 4601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:54657: BlockingRecv error: recv got EOF from 127.30.137.193:54657 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 05:36:09 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:09 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:09.686193 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:09.677613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:34517)
0809 05:36:09.678206 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:09.678224 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:09.678346 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:09.678627 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:09.678630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:09.679103 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:09.679294 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:09.679305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.680325 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.680330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:09.681316 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:09.681330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.682033 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.682038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:09.682158 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:09.682814 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:09.682832 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:09.685820 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":307,"threads_started":1}
W20260809 05:36:09.686331 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:09.677794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:34517 (local address 127.30.137.254:35515)
0809 05:36:09.677964 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:09.677968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:09.678304 (+ 336us) server_negotiation.cc:407] Connection header received
0809 05:36:09.678449 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:09.678453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:09.678530 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:09.678613 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:09.679422 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.680166 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.681448 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:09.681872 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:09.681963 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:09.686063 (+ 4100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:34517: BlockingRecv error: recv got EOF from 127.30.137.194:34517 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 05:36:10 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:10 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:10.153429 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:10.144098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:54407)
0809 05:36:10.144725 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:10.144746 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:10.144900 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:10.145235 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:10.145239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:10.145649 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:10.145826 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:10.145835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.146949 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.146958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:10.147994 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:10.148009 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.149069 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.149077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:10.149273 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:10.149989 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:10.150019 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:10.153169 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":357,"threads_started":1}
W20260809 05:36:10.153708 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:10.144255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:54407 (local address 127.30.137.254:35515)
0809 05:36:10.144450 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:10.144455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:10.144891 (+ 436us) server_negotiation.cc:407] Connection header received
0809 05:36:10.145017 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:10.145022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:10.145091 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:10.145195 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:10.145970 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.146751 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.148166 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.148855 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.148996 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:10.153427 (+ 4431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:54407: BlockingRecv error: recv got EOF from 127.30.137.195:54407 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 05:36:10 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:10 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:10.208608 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:10.199570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:47851)
0809 05:36:10.200165 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:10.200190 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:10.200357 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:10.200750 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:10.200754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:10.201195 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:10.201379 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:10.201387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.202462 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.202468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:10.203459 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:10.203470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.204214 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.204220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:10.204395 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:10.205061 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:10.205079 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:10.208192 (+ 3113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":313,"threads_started":1}
W20260809 05:36:10.208689 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:10.199735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:47851 (local address 127.30.137.254:35515)
0809 05:36:10.200079 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:10.200085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:10.200281 (+ 196us) server_negotiation.cc:407] Connection header received
0809 05:36:10.200486 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:10.200492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:10.200597 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:10.200717 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:10.201533 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.202310 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.203604 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.204050 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.204192 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:10.208424 (+ 4232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:47851: BlockingRecv error: recv got EOF from 127.30.137.193:47851 (error 108)
Metrics: {"server-negotiator.queue_time_us":195}
Aug 09 05:36:10 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:10 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:10.701290 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:10.687346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:48317)
0809 05:36:10.687890 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:10.687909 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:10.688036 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:10.688365 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:10.688368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:10.688769 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:10.688957 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:10.688966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.690005 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.690008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:10.691299 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:10.691320 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.692047 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.692052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:10.692176 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:10.697875 (+ 5699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:10.697897 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:10.701016 (+ 3119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":304,"threads_started":1}
W20260809 05:36:10.701583 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:10.687472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:48317 (local address 127.30.137.254:35515)
0809 05:36:10.687655 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:10.687660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:10.688013 (+ 353us) server_negotiation.cc:407] Connection header received
0809 05:36:10.688148 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:10.688153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:10.688231 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:10.688319 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:10.689094 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.689872 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.691456 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:10.691898 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:10.691999 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:10.701282 (+ 9283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:48317: BlockingRecv error: recv got EOF from 127.30.137.194:48317 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 05:36:11 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:11 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.195@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:11.163722 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:11.154472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.195:48737)
0809 05:36:11.155120 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:11.155144 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:11.155324 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:11.155781 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:11.155786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:11.156317 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:11.156577 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:11.156592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.157737 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.157742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:11.158901 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:11.158914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.159611 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.159616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:11.159754 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:11.160406 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:11.160424 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:11.163442 (+ 3018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":351,"threads_started":1}
W20260809 05:36:11.163946 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:11.154711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:48737 (local address 127.30.137.254:35515)
0809 05:36:11.154975 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:11.154981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:11.155273 (+ 292us) server_negotiation.cc:407] Connection header received
0809 05:36:11.155453 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:11.155460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:11.155565 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:11.155680 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:11.156735 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.157588 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.159038 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.159458 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.159557 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:11.163697 (+ 4140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.195:48737: BlockingRecv error: recv got EOF from 127.30.137.195:48737 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 09 05:36:11 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:11 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.193@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:11.219569 13060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:11.209974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.193:38193)
0809 05:36:11.210595 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:11.210613 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:11.210832 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:11.211218 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:11.211223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:11.211681 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:11.211870 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:11.211879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.212958 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.212964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:11.213897 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:11.213907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.214711 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.214717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:11.214915 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:11.215610 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:11.215628 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:11.219308 (+ 3680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":366,"threads_started":1}
W20260809 05:36:11.219733 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:11.210171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:38193 (local address 127.30.137.254:35515)
0809 05:36:11.210372 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:11.210376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:11.210726 (+ 350us) server_negotiation.cc:407] Connection header received
0809 05:36:11.210979 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:11.210992 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:11.211056 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:11.211165 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:11.212021 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.212803 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.214044 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.214514 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.214661 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:11.219495 (+ 4834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.193:38193: BlockingRecv error: recv got EOF from 127.30.137.193:38193 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 05:36:11 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 05:36:11 dist-test-slave-3kk6 krb5kdc[11538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.137.194@KRBTEST.COM for oryx/127.30.137.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 05:36:11.711326 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:11.702579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35515 (local address 127.30.137.194:39867)
0809 05:36:11.703263 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:11.703281 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:11.703398 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:11.703707 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:11.703710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:11.704124 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:11.704316 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:11.704325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.705361 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.705365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:11.706322 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:11.706335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.707079 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.707083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:11.707215 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:11.707885 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:11.707902 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:11.711053 (+ 3151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.137.254:35515: Server oryx/127.30.137.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":428,"threads_started":1}
W20260809 05:36:11.711575 12626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:11.702691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:39867 (local address 127.30.137.254:35515)
0809 05:36:11.702900 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:11.702905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:11.703354 (+ 449us) server_negotiation.cc:407] Connection header received
0809 05:36:11.703495 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:11.703498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:11.703561 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:11.703643 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:11.704452 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.705219 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.706447 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:11.706943 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:11.707047 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:11.711281 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.137.194:39867: BlockingRecv error: recv got EOF from 127.30.137.194:39867 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 05:36:11.996207 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12410
I20260809 05:36:12.009728 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:11.997419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.30.137.193:44305)
0809 05:36:11.998052 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:11.998059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:11.998086 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:36:11.998152 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:11.998158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:11.998401 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:11.998565 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:12.000219 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.001287 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.002701 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.003327 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.003444 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:12.006323 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:12.006332 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:12.006347 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:12.006385 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:12.009176 (+ 2791us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 05:36:12.009482 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48980: Request ticket server kudu/127.30.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":334,"threads_started":1}
W20260809 05:36:12.009888 13063 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48980: Request ticket server kudu/127.30.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 05:36:12.010020 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12410: Not authorized: Client connection negotiation failed: client connection to 127.30.137.193:44305: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.137.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 05:36:12.010280 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 12410
I20260809 05:36:12.036950 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12630
I20260809 05:36:12.050356 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:12.038177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.30.137.194:34711)
0809 05:36:12.038886 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:12.038893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:12.038927 (+ 34us) server_negotiation.cc:407] Connection header received
0809 05:36:12.038994 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:12.039002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:12.039236 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:12.039415 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:12.040773 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.041953 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.043518 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.044166 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.044282 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:12.046946 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:12.046956 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:12.046971 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:12.047007 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:12.049777 (+ 2770us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 05:36:12.050066 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54268: Request ticket server kudu/127.30.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":345,"threads_started":1}
W20260809 05:36:12.050489 13064 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54268: Request ticket server kudu/127.30.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 05:36:12.050535 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12630: Not authorized: Client connection negotiation failed: client connection to 127.30.137.194:34711: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.137.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 05:36:12.050767 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 12630
I20260809 05:36:12.078012 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12810
I20260809 05:36:12.091589 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:12.079255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.30.137.195:35315)
0809 05:36:12.080043 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:12.080051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:12.080082 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:36:12.080154 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:12.080161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:12.080402 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:12.080574 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:12.081975 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.083295 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.084605 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.085244 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.085356 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:12.088201 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:12.088211 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:12.088225 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:12.088271 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:12.091033 (+ 2762us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 05:36:12.091325 (+ 292us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59602: Request ticket server kudu/127.30.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":462,"threads_started":1}
W20260809 05:36:12.091747 13065 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59602: Request ticket server kudu/127.30.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 05:36:12.091756 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12810: Not authorized: Client connection negotiation failed: client connection to 127.30.137.195:35315: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.137.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 05:36:12.091936 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 12810
I20260809 05:36:12.121721 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 12316
I20260809 05:36:12.123411 12401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:12.245090 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 12316
2026-08-09T05:36:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21891 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T05:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:36:12Z Disabled control of system clock
I20260809 05:36:12.349933 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:36775
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:36775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 05:36:12.714839 13072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:12.715164 13072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:12.715322 13072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:12.723107 13072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:36:12.723256 13072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:12.723333 13072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:12.723439 13072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:36:12.723529 13072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:36:12.733175 13072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:36775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:36775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:12.734571 13072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:12.736445 13072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:12.748631 13078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:12.748700 13080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:12.748911 13077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:12.749445 13072 server_base.cc:1034] running on GCE node
I20260809 05:36:12.750260 13072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:12.751780 13072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:12.753037 13072 hybrid_clock.cc:648] HybridClock initialized: now 1786253772752997 us; error 57 us; skew 500 ppm
I20260809 05:36:12.753403 13072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:12.755879 13072 webserver.cc:460] Webserver started at http://127.30.137.254:44609/ using document root <none> and password file <none>
I20260809 05:36:12.756723 13072 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:12.756870 13072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:12.757222 13072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:12.760385 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "992def7e61944a48896ab93d1801f742"
format_stamp: "Formatted at 2026-08-09 05:36:12 on dist-test-slave-3kk6"
server_key: "28da633554afa185231fbc482adec6d7"
server_key_iv: "6fb9f40955331a13c41f2745b58646ce"
server_key_version: "encryptionkey@0"
I20260809 05:36:12.761255 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "992def7e61944a48896ab93d1801f742"
format_stamp: "Formatted at 2026-08-09 05:36:12 on dist-test-slave-3kk6"
server_key: "28da633554afa185231fbc482adec6d7"
server_key_iv: "6fb9f40955331a13c41f2745b58646ce"
server_key_version: "encryptionkey@0"
I20260809 05:36:12.766927 13072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:36:12.770577 13089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:12.772049 13072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:36:12.772260 13072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "992def7e61944a48896ab93d1801f742"
format_stamp: "Formatted at 2026-08-09 05:36:12 on dist-test-slave-3kk6"
server_key: "28da633554afa185231fbc482adec6d7"
server_key_iv: "6fb9f40955331a13c41f2745b58646ce"
server_key_version: "encryptionkey@0"
I20260809 05:36:12.772504 13072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:12.798398 13072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:12.799427 13072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:12.799784 13072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:12.819530 13072 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:36775
I20260809 05:36:12.819553 13166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:36775 every 8 connection(s)
I20260809 05:36:12.821002 13072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:36:12.821995 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 13072
I20260809 05:36:12.822220 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:36:12.823000 31271 external_mini_cluster.cc:1428] Setting key 02f0491f7e858baf0935966200f4ecfd
I20260809 05:36:12.826058 13167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:36:12.837410 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:12.824902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.30.137.254:36775)
0809 05:36:12.825905 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:12.825916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:12.825958 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:36:12.827719 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:12.827742 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:12.828184 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:12.828533 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:12.829275 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.830624 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.832711 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:12.833367 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:12.833502 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:12.834953 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:12.834962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:12.834976 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:12.835566 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:12.835640 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:12.835665 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:12.835791 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:12.836024 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 05:36:12.836388 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":380,"threads_started":1}
I20260809 05:36:12.839254 13167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742: Bootstrap starting.
I20260809 05:36:12.842442 13167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742: Neither blocks nor log segments found. Creating new log.
I20260809 05:36:12.843528 13167 log.cc:824] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742: Log is configured to *not* fsync() on all Append() calls
I20260809 05:36:12.846617 13167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742: No bootstrap required, opened a new log
I20260809 05:36:12.854238 13167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992def7e61944a48896ab93d1801f742" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 36775 } }
I20260809 05:36:12.854712 13167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:36:12.854830 13167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 992def7e61944a48896ab93d1801f742, State: Initialized, Role: FOLLOWER
I20260809 05:36:12.855448 13167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992def7e61944a48896ab93d1801f742" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 36775 } }
I20260809 05:36:12.855695 13167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:36:12.855850 13167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:36:12.856035 13167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:36:12.858709 13167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992def7e61944a48896ab93d1801f742" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 36775 } }
I20260809 05:36:12.859326 13167 leader_election.cc:302] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 992def7e61944a48896ab93d1801f742; no voters:
I20260809 05:36:12.859954 13167 leader_election.cc:288] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:36:12.860180 13178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:36:12.861094 13178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [term 1 LEADER]: Becoming Leader. State: Replica: 992def7e61944a48896ab93d1801f742, State: Running, Role: LEADER
I20260809 05:36:12.861668 13178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992def7e61944a48896ab93d1801f742" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 36775 } }
I20260809 05:36:12.862223 13167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:36:12.864846 13179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "992def7e61944a48896ab93d1801f742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992def7e61944a48896ab93d1801f742" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 36775 } } }
I20260809 05:36:12.865329 13179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [sys.catalog]: This master's current role is: LEADER
I20260809 05:36:12.865864 13180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 992def7e61944a48896ab93d1801f742. Latest consensus state: current_term: 1 leader_uuid: "992def7e61944a48896ab93d1801f742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "992def7e61944a48896ab93d1801f742" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 36775 } } }
I20260809 05:36:12.866240 13180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742 [sys.catalog]: This master's current role is: LEADER
I20260809 05:36:12.868481 13188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:36:12.873369 13188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:36:12.881897 13188 catalog_manager.cc:1347] Generated new cluster ID: ed4f85f08bbc4c789f8d5df606d181ab
I20260809 05:36:12.882036 13188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:36:12.895977 13188 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:36:12.897346 13188 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:36:12.908175 13188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 992def7e61944a48896ab93d1801f742: Generated new TSK 0
I20260809 05:36:12.909097 13188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:36:12.921797 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:36775
--builtin_ntp_servers=127.30.137.212:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 05:36:13.280927 13204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:13.281227 13204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:13.281386 13204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:13.288931 13204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:13.289050 13204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:13.289234 13204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:36:13.298697 13204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:36775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:13.300072 13204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:13.301945 13204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:13.315572 13212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:13.315585 13215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:13.315618 13211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:13.316336 13204 server_base.cc:1034] running on GCE node
I20260809 05:36:13.316931 13204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:13.317885 13204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:13.319094 13204 hybrid_clock.cc:648] HybridClock initialized: now 1786253773319057 us; error 27 us; skew 500 ppm
I20260809 05:36:13.319393 13204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:13.321589 13204 webserver.cc:460] Webserver started at http://127.30.137.193:46405/ using document root <none> and password file <none>
I20260809 05:36:13.322254 13204 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:13.322345 13204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:13.322625 13204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:13.325522 13204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "ef2eb5ce12b6414e9cc288e15f8e0dee"
format_stamp: "Formatted at 2026-08-09 05:36:13 on dist-test-slave-3kk6"
server_key: "129bcdfedd5f815ba290360baa426f6c"
server_key_iv: "0999a247f08decbeaf80a4ff69ccd09c"
server_key_version: "encryptionkey@0"
I20260809 05:36:13.326292 13204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "ef2eb5ce12b6414e9cc288e15f8e0dee"
format_stamp: "Formatted at 2026-08-09 05:36:13 on dist-test-slave-3kk6"
server_key: "129bcdfedd5f815ba290360baa426f6c"
server_key_iv: "0999a247f08decbeaf80a4ff69ccd09c"
server_key_version: "encryptionkey@0"
I20260809 05:36:13.331487 13204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 05:36:13.335044 13222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:13.336407 13204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 05:36:13.336603 13204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "ef2eb5ce12b6414e9cc288e15f8e0dee"
format_stamp: "Formatted at 2026-08-09 05:36:13 on dist-test-slave-3kk6"
server_key: "129bcdfedd5f815ba290360baa426f6c"
server_key_iv: "0999a247f08decbeaf80a4ff69ccd09c"
server_key_version: "encryptionkey@0"
I20260809 05:36:13.336838 13204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:13.351115 13204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:13.351972 13204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:13.352295 13204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:13.353340 13204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:13.354900 13204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:13.355031 13204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:13.355158 13204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:13.355254 13204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:13.390203 13204 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:39915
I20260809 05:36:13.390249 13368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:39915 every 8 connection(s)
I20260809 05:36:13.391691 13204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:36:13.394214 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 13204
I20260809 05:36:13.394433 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:36:13.395345 31271 external_mini_cluster.cc:1428] Setting key 38b1e7d4f775ab7188ba1c2180684546
I20260809 05:36:13.398339 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:36775
--builtin_ntp_servers=127.30.137.212:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 05:36:13.405915 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:13.394522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:58711 (local address 127.30.137.254:36775)
0809 05:36:13.395045 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:13.395049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:13.396491 (+ 1442us) server_negotiation.cc:407] Connection header received
0809 05:36:13.398360 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:13.398367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:13.398454 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:13.398599 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:13.399742 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.400934 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.402551 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.403316 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.403467 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:13.404852 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:13.404861 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:13.404868 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:13.404895 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:13.404961 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:13.404981 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:13.405080 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:13.405364 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 05:36:13.405485 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":282,"threads_started":1}
I20260809 05:36:13.406896 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:13.395276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:36775 (local address 127.30.137.193:58711)
0809 05:36:13.396276 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:13.396347 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:13.398074 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:13.398870 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:13.398877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:13.398959 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:13.399562 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:13.399580 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.401098 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.401102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:13.402378 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:13.402389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.403544 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.403550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:13.403754 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:13.404522 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:36:13.404524 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:13.404565 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:36:13.404646 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:13.405114 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:13.405124 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:13.405131 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 05:36:13.405325 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 05:36:13.405636 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":321,"threads_started":1}
I20260809 05:36:13.408696 13369 heartbeater.cc:347] Connected to a master server at 127.30.137.254:36775
I20260809 05:36:13.409047 13369 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:13.409873 13369 heartbeater.cc:511] Master 127.30.137.254:36775 requested a full tablet report, sending...
I20260809 05:36:13.412245 13115 ts_manager.cc:194] Registered new tserver with Master: ef2eb5ce12b6414e9cc288e15f8e0dee (127.30.137.193:39915)
I20260809 05:36:13.414407 13115 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.193:58711
W20260809 05:36:13.750084 13378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:13.750375 13378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:13.750463 13378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:13.757725 13378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:13.757838 13378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:13.757946 13378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:36:13.767781 13378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:36775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:13.769109 13378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:13.770978 13378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:13.784857 13384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:13.784762 13385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:13.784799 13387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:13.785745 13378 server_base.cc:1034] running on GCE node
I20260809 05:36:13.786420 13378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:13.787595 13378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:13.788841 13378 hybrid_clock.cc:648] HybridClock initialized: now 1786253773788782 us; error 52 us; skew 500 ppm
I20260809 05:36:13.789191 13378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:13.792960 13378 webserver.cc:460] Webserver started at http://127.30.137.194:42901/ using document root <none> and password file <none>
I20260809 05:36:13.793725 13378 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:13.793895 13378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:13.794250 13378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:13.797348 13378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "986b430f9fdc4e1489f1305d934034bd"
format_stamp: "Formatted at 2026-08-09 05:36:13 on dist-test-slave-3kk6"
server_key: "e3efb60574b8a1e04a0f7085d0da9e8b"
server_key_iv: "e6e3ae215462f543e4bee8ad27d5a428"
server_key_version: "encryptionkey@0"
I20260809 05:36:13.798269 13378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "986b430f9fdc4e1489f1305d934034bd"
format_stamp: "Formatted at 2026-08-09 05:36:13 on dist-test-slave-3kk6"
server_key: "e3efb60574b8a1e04a0f7085d0da9e8b"
server_key_iv: "e6e3ae215462f543e4bee8ad27d5a428"
server_key_version: "encryptionkey@0"
I20260809 05:36:13.803735 13378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 05:36:13.807520 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:13.808902 13378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:36:13.809106 13378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "986b430f9fdc4e1489f1305d934034bd"
format_stamp: "Formatted at 2026-08-09 05:36:13 on dist-test-slave-3kk6"
server_key: "e3efb60574b8a1e04a0f7085d0da9e8b"
server_key_iv: "e6e3ae215462f543e4bee8ad27d5a428"
server_key_version: "encryptionkey@0"
I20260809 05:36:13.809357 13378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:13.825237 13378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:13.826125 13378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:13.826455 13378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:13.827638 13378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:13.829160 13378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:13.829288 13378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:13.829432 13378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:13.829523 13378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:13.866150 13378 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:40253
I20260809 05:36:13.866205 13554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:40253 every 8 connection(s)
I20260809 05:36:13.867803 13378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:36:13.871922 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 13378
I20260809 05:36:13.872102 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:36:13.872654 31271 external_mini_cluster.cc:1428] Setting key c9c59c2f5e928bca60255aaffaf0b4a1
I20260809 05:36:13.875854 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:36775
--builtin_ntp_servers=127.30.137.212:37543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 05:36:13.882364 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:13.870524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:35153 (local address 127.30.137.254:36775)
0809 05:36:13.870777 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:13.870782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:13.872454 (+ 1672us) server_negotiation.cc:407] Connection header received
0809 05:36:13.874794 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:13.874800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:13.874901 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:13.875003 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:13.876253 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.877521 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.879210 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.879900 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.880046 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:13.881423 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:13.881431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:13.881439 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:13.881467 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:13.881510 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:13.881529 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:13.881625 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:13.881829 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 05:36:13.881948 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:36:13.883277 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:13.871100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:36775 (local address 127.30.137.194:35153)
0809 05:36:13.872276 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:13.872319 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:13.874416 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:13.875214 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:13.875222 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:13.875307 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:13.875991 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:13.876021 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.877677 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.877681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:13.879065 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:13.879078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:13.880114 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:13.880119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:13.880323 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:13.881085 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:36:13.881088 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:13.881137 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:36:13.881218 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:13.881628 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:13.881636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:13.881642 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:36:13.881787 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 05:36:13.882111 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":401,"threads_started":1}
I20260809 05:36:13.884840 13556 heartbeater.cc:347] Connected to a master server at 127.30.137.254:36775
I20260809 05:36:13.885196 13556 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:13.886219 13556 heartbeater.cc:511] Master 127.30.137.254:36775 requested a full tablet report, sending...
I20260809 05:36:13.887969 13115 ts_manager.cc:194] Registered new tserver with Master: 986b430f9fdc4e1489f1305d934034bd (127.30.137.194:40253)
I20260809 05:36:13.888824 13115 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.194:35153
W20260809 05:36:14.238023 13560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:14.238412 13560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:14.238577 13560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:14.246065 13560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:14.246227 13560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:14.246423 13560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:36:14.255887 13560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:36775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:14.257275 13560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:14.259203 13560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:14.273545 13572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:14.273567 13566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:14.273653 13568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:14.274008 13560 server_base.cc:1034] running on GCE node
I20260809 05:36:14.274995 13560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:14.276317 13560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:14.277590 13560 hybrid_clock.cc:648] HybridClock initialized: now 1786253774277487 us; error 60 us; skew 500 ppm
I20260809 05:36:14.278007 13560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:14.280668 13560 webserver.cc:460] Webserver started at http://127.30.137.195:41417/ using document root <none> and password file <none>
I20260809 05:36:14.281430 13560 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:14.281559 13560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:14.281919 13560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:14.285048 13560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "d355b5b99d4740fdae27e7c794f05eec"
format_stamp: "Formatted at 2026-08-09 05:36:14 on dist-test-slave-3kk6"
server_key: "999b1713525aa1e4a57388654a3bb373"
server_key_iv: "1d70fd2387253bd1b1fc7b1e36263f3e"
server_key_version: "encryptionkey@0"
I20260809 05:36:14.285918 13560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "d355b5b99d4740fdae27e7c794f05eec"
format_stamp: "Formatted at 2026-08-09 05:36:14 on dist-test-slave-3kk6"
server_key: "999b1713525aa1e4a57388654a3bb373"
server_key_iv: "1d70fd2387253bd1b1fc7b1e36263f3e"
server_key_version: "encryptionkey@0"
I20260809 05:36:14.291531 13560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 05:36:14.295177 13579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:14.296478 13560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 05:36:14.296653 13560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "d355b5b99d4740fdae27e7c794f05eec"
format_stamp: "Formatted at 2026-08-09 05:36:14 on dist-test-slave-3kk6"
server_key: "999b1713525aa1e4a57388654a3bb373"
server_key_iv: "1d70fd2387253bd1b1fc7b1e36263f3e"
server_key_version: "encryptionkey@0"
I20260809 05:36:14.296893 13560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:14.323237 13560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:14.324183 13560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:14.324523 13560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:14.325559 13560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:14.327180 13560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:14.327309 13560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:14.327440 13560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:14.327535 13560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:14.362881 13560 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:41927
I20260809 05:36:14.362998 13750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:41927 every 8 connection(s)
I20260809 05:36:14.364391 13560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:36:14.368829 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 13560
I20260809 05:36:14.369014 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:36:14.369488 31271 external_mini_cluster.cc:1428] Setting key b3b13d3978708bce8f59a24f60119959
I20260809 05:36:14.378235 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:14.367283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:49163 (local address 127.30.137.254:36775)
0809 05:36:14.367475 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:14.367479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:14.369030 (+ 1551us) server_negotiation.cc:407] Connection header received
0809 05:36:14.371435 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:14.371442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:14.371530 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:14.371652 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:14.373034 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.373773 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.375497 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.375988 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.376095 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:14.377549 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:14.377554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:14.377557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:14.377577 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:14.377608 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:14.377621 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:14.377692 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:14.377901 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 05:36:14.377981 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:36:14.379444 13756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:14.367859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:36775 (local address 127.30.137.195:49163)
0809 05:36:14.368839 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:14.368884 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:14.371111 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:14.371926 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:14.371940 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:14.372069 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:14.372877 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:14.372897 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.373927 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.373930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:14.375342 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:14.375356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.376256 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.376262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:14.376482 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:14.377186 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:36:14.377189 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:14.377229 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:36:14.377358 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:14.377708 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:14.377716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:14.377721 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 05:36:14.377872 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 05:36:14.378196 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":375,"threads_started":1}
I20260809 05:36:14.380954 13753 heartbeater.cc:347] Connected to a master server at 127.30.137.254:36775
I20260809 05:36:14.381273 13753 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:14.382047 13753 heartbeater.cc:511] Master 127.30.137.254:36775 requested a full tablet report, sending...
I20260809 05:36:14.383756 13115 ts_manager.cc:194] Registered new tserver with Master: d355b5b99d4740fdae27e7c794f05eec (127.30.137.195:41927)
I20260809 05:36:14.384747 13115 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.195:49163
I20260809 05:36:14.385418 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 05:36:14.392879 13373 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:14.391116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.30.137.254:36775)
0809 05:36:14.391395 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:14.391402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:14.391782 (+ 380us) server_negotiation.cc:407] Connection header received
0809 05:36:14.391971 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:14.391975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:14.392047 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:14.392130 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:14.392619 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59848: BlockingRecv error: recv got EOF from 127.0.0.1:59848 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:36:14.393428 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 13204
I20260809 05:36:14.402534 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:14.394534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.30.137.193:39915)
0809 05:36:14.395168 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:14.395177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:14.395532 (+ 355us) server_negotiation.cc:407] Connection header received
0809 05:36:14.395634 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:14.395643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:14.395914 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:14.396068 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:14.396435 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.397887 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.399185 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.399730 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.399833 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:14.400978 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:14.400987 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:14.400999 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:14.401803 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:14.401850 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:14.401871 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:14.401970 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:14.402113 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 05:36:14.402229 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":311,"threads_started":1}
I20260809 05:36:14.403805 13364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:14.487685 13369 heartbeater.cc:503] Master 127.30.137.254:36775 was elected leader, sending a full tablet report...
I20260809 05:36:14.488832 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 13204
W20260809 05:36:14.512843 13093 connection.cc:535] server connection from 127.30.137.193:58711 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 05:36:14.513211 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 13378
I20260809 05:36:14.521150 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:14.514221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.30.137.194:40253)
0809 05:36:14.514883 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:14.514890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:14.514918 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:36:14.514994 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:14.515000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:14.515231 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:14.515371 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:14.515786 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.516837 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.518079 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.518634 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.518837 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:14.519769 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:14.519777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:14.519791 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:14.520402 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:14.520455 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:14.520475 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:14.520564 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:14.520698 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 05:36:14.520840 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":332,"threads_started":1}
I20260809 05:36:14.522559 13548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:14.606117 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 13378
I20260809 05:36:14.631115 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 13560
I20260809 05:36:14.640067 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:14.632181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.30.137.195:41927)
0809 05:36:14.632922 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:14.632929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:14.632960 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:36:14.633023 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:14.633029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:14.633262 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:14.633437 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:14.633957 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.635151 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.636443 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:14.637041 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:14.637200 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:14.638434 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:14.638444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:14.638459 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:14.639161 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:14.639213 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:14.639234 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:14.639361 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:14.639550 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 05:36:14.639703 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":382,"threads_started":1}
I20260809 05:36:14.641467 13736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:14.725678 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 13560
I20260809 05:36:14.752250 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 13072
I20260809 05:36:14.753510 13159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:14.858901 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 13072
2026-08-09T05:36:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2600 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T05:36:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:36:14Z Disabled control of system clock
I20260809 05:36:14.950911 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:33073
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:33073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 05:36:15.309232 13786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:15.309592 13786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:15.309772 13786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:15.317525 13786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:36:15.317693 13786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:15.317775 13786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:15.317879 13786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:36:15.317966 13786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:36:15.327647 13786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:33073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:33073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:15.329073 13786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:15.330972 13786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:15.343698 13799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:15.343791 13795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:15.343765 13797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:15.344830 13786 server_base.cc:1034] running on GCE node
I20260809 05:36:15.345700 13786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:15.347230 13786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:15.348454 13786 hybrid_clock.cc:648] HybridClock initialized: now 1786253775348415 us; error 54 us; skew 500 ppm
I20260809 05:36:15.348790 13786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:15.351333 13786 webserver.cc:460] Webserver started at http://127.30.137.254:36723/ using document root <none> and password file <none>
I20260809 05:36:15.352051 13786 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:15.352142 13786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:15.352432 13786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:15.355602 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "1537eb0859dd421abe7fefc574dbaea3"
format_stamp: "Formatted at 2026-08-09 05:36:15 on dist-test-slave-3kk6"
server_key: "204aee189eb50766276237b002b1c3b6"
server_key_iv: "44d56abd1dd93c01393fadc5934c9e23"
server_key_version: "encryptionkey@0"
I20260809 05:36:15.356426 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "1537eb0859dd421abe7fefc574dbaea3"
format_stamp: "Formatted at 2026-08-09 05:36:15 on dist-test-slave-3kk6"
server_key: "204aee189eb50766276237b002b1c3b6"
server_key_iv: "44d56abd1dd93c01393fadc5934c9e23"
server_key_version: "encryptionkey@0"
I20260809 05:36:15.361820 13786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:36:15.365576 13807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:15.367389 13786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:36:15.367579 13786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "1537eb0859dd421abe7fefc574dbaea3"
format_stamp: "Formatted at 2026-08-09 05:36:15 on dist-test-slave-3kk6"
server_key: "204aee189eb50766276237b002b1c3b6"
server_key_iv: "44d56abd1dd93c01393fadc5934c9e23"
server_key_version: "encryptionkey@0"
I20260809 05:36:15.367843 13786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:15.374032 13786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:15.374926 13786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:15.375237 13786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:15.393308 13786 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:33073
I20260809 05:36:15.393332 13872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:33073 every 8 connection(s)
I20260809 05:36:15.394868 13786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:36:15.399084 13874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:36:15.403321 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 13786
I20260809 05:36:15.403568 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:36:15.404316 31271 external_mini_cluster.cc:1428] Setting key 0a60c432b49f2d4c0d481d9a289be99c
I20260809 05:36:15.412755 13874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3: Bootstrap starting.
I20260809 05:36:15.415126 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:15.406551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.30.137.254:33073)
0809 05:36:15.407791 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:15.407811 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:15.407867 (+ 56us) server_negotiation.cc:407] Connection header received
0809 05:36:15.410592 (+ 2725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:15.410627 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:15.411293 (+ 666us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:15.411822 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:15.412932 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:15.412940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:15.412957 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:15.412975 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:15.413060 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:15.413086 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:15.413327 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:15.413468 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 05:36:15.413754 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":360,"threads_started":1}
I20260809 05:36:15.416831 13874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3: Neither blocks nor log segments found. Creating new log.
I20260809 05:36:15.418222 13874 log.cc:824] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3: Log is configured to *not* fsync() on all Append() calls
I20260809 05:36:15.421262 13874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3: No bootstrap required, opened a new log
I20260809 05:36:15.428684 13874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1537eb0859dd421abe7fefc574dbaea3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33073 } }
I20260809 05:36:15.429093 13874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:36:15.429204 13874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1537eb0859dd421abe7fefc574dbaea3, State: Initialized, Role: FOLLOWER
I20260809 05:36:15.429842 13874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1537eb0859dd421abe7fefc574dbaea3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33073 } }
I20260809 05:36:15.430095 13874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:36:15.430254 13874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:36:15.430438 13874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:36:15.432955 13874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1537eb0859dd421abe7fefc574dbaea3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33073 } }
I20260809 05:36:15.433467 13874 leader_election.cc:302] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1537eb0859dd421abe7fefc574dbaea3; no voters:
I20260809 05:36:15.434084 13874 leader_election.cc:288] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:36:15.434226 13881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:36:15.434979 13881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [term 1 LEADER]: Becoming Leader. State: Replica: 1537eb0859dd421abe7fefc574dbaea3, State: Running, Role: LEADER
I20260809 05:36:15.435488 13881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1537eb0859dd421abe7fefc574dbaea3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33073 } }
I20260809 05:36:15.436211 13874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:36:15.438616 13885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1537eb0859dd421abe7fefc574dbaea3. Latest consensus state: current_term: 1 leader_uuid: "1537eb0859dd421abe7fefc574dbaea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1537eb0859dd421abe7fefc574dbaea3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33073 } } }
I20260809 05:36:15.438668 13884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1537eb0859dd421abe7fefc574dbaea3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1537eb0859dd421abe7fefc574dbaea3" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 33073 } } }
I20260809 05:36:15.439098 13885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [sys.catalog]: This master's current role is: LEADER
I20260809 05:36:15.439095 13884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3 [sys.catalog]: This master's current role is: LEADER
I20260809 05:36:15.441174 13889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:36:15.446462 13889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:36:15.456022 13889 catalog_manager.cc:1347] Generated new cluster ID: 86df29791c1a42dfb78913411297218c
I20260809 05:36:15.456164 13889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:36:15.479619 13889 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:36:15.480149 13889 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:36:15.487543 13889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1537eb0859dd421abe7fefc574dbaea3: Generated new TSK 0
I20260809 05:36:15.488482 13889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:36:15.499490 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33073
--builtin_ntp_servers=127.30.137.212:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 05:36:15.861642 13905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:15.861963 13905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:15.862128 13905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:15.869918 13905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:15.870038 13905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:15.870214 13905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:36:15.879943 13905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:15.881320 13905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:15.883399 13905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:15.897655 13915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:15.897706 13917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:15.897693 13912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:15.898893 13905 server_base.cc:1034] running on GCE node
I20260809 05:36:15.899648 13905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:15.900712 13905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:15.901958 13905 hybrid_clock.cc:648] HybridClock initialized: now 1786253775901892 us; error 67 us; skew 500 ppm
I20260809 05:36:15.902320 13905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:15.909493 13905 webserver.cc:460] Webserver started at http://127.30.137.193:34741/ using document root <none> and password file <none>
I20260809 05:36:15.910362 13905 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:15.910521 13905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:15.910948 13905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:15.914223 13905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "76bd8b4f111b43f296cc587933c41ce2"
format_stamp: "Formatted at 2026-08-09 05:36:15 on dist-test-slave-3kk6"
server_key: "96a44b0f2b8555b5fe24b03c564d7db9"
server_key_iv: "2e9d53fa03ef4290af5209208df09b39"
server_key_version: "encryptionkey@0"
I20260809 05:36:15.915112 13905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "76bd8b4f111b43f296cc587933c41ce2"
format_stamp: "Formatted at 2026-08-09 05:36:15 on dist-test-slave-3kk6"
server_key: "96a44b0f2b8555b5fe24b03c564d7db9"
server_key_iv: "2e9d53fa03ef4290af5209208df09b39"
server_key_version: "encryptionkey@0"
I20260809 05:36:15.920694 13905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:36:15.924404 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:15.925822 13905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:36:15.926015 13905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "76bd8b4f111b43f296cc587933c41ce2"
format_stamp: "Formatted at 2026-08-09 05:36:15 on dist-test-slave-3kk6"
server_key: "96a44b0f2b8555b5fe24b03c564d7db9"
server_key_iv: "2e9d53fa03ef4290af5209208df09b39"
server_key_version: "encryptionkey@0"
I20260809 05:36:15.926262 13905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:15.932267 13905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:15.933109 13905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:15.933410 13905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:15.934357 13905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:15.935883 13905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:15.936015 13905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:15.936158 13905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:15.936249 13905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:15.971987 13905 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:39359
I20260809 05:36:15.972008 14085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:39359 every 8 connection(s)
I20260809 05:36:15.973593 13905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:36:15.982031 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 13905
I20260809 05:36:15.982183 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:15.976370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:51287 (local address 127.30.137.254:33073)
0809 05:36:15.976850 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:15.976854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:15.978099 (+ 1245us) server_negotiation.cc:407] Connection header received
0809 05:36:15.979902 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:15.979906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:15.979965 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:15.980071 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:15.981380 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:15.981384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:15.981388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:15.981405 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:15.981449 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:15.981460 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:15.981567 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:15.981843 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 05:36:15.981898 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":249,"threads_started":1}
I20260809 05:36:15.982267 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:36:15.983180 31271 external_mini_cluster.cc:1428] Setting key bc8e612501af7f9fd40e9a167c675793
I20260809 05:36:15.983577 14089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:15.977001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33073 (local address 127.30.137.193:51287)
0809 05:36:15.977926 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:15.977971 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:15.979637 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:15.980292 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:15.980311 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:15.980403 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:15.981076 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:36:15.981078 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:15.981117 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:36:15.981203 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:15.981576 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:15.981587 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:15.981596 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:36:15.981943 (+ 347us) client_negotiation.cc:240] Negotiation successful
0809 05:36:15.982244 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":277,"threads_started":1}
I20260809 05:36:15.985776 14086 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33073
I20260809 05:36:15.986171 14086 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:15.986809 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33073
--builtin_ntp_servers=127.30.137.212:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 05:36:15.987239 14086 heartbeater.cc:511] Master 127.30.137.254:33073 requested a full tablet report, sending...
I20260809 05:36:15.989948 13828 ts_manager.cc:194] Registered new tserver with Master: 76bd8b4f111b43f296cc587933c41ce2 (127.30.137.193:39359)
W20260809 05:36:16.353390 14090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:16.353766 14090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:16.353935 14090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:16.361471 14090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:16.361670 14090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:16.361868 14090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:36:16.371721 14090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:16.373165 14090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:16.375113 14090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:16.389146 14101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:16.389186 14105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:16.388967 14103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:16.390162 14090 server_base.cc:1034] running on GCE node
I20260809 05:36:16.390897 14090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:16.391938 14090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:16.393203 14090 hybrid_clock.cc:648] HybridClock initialized: now 1786253776393147 us; error 60 us; skew 500 ppm
I20260809 05:36:16.393620 14090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:16.396627 14090 webserver.cc:460] Webserver started at http://127.30.137.194:44745/ using document root <none> and password file <none>
I20260809 05:36:16.397521 14090 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:16.397634 14090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:16.398005 14090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:16.402065 14090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "4007cb2cf31c45a597604e4b7ad55485"
format_stamp: "Formatted at 2026-08-09 05:36:16 on dist-test-slave-3kk6"
server_key: "47d10368592f1b13008c30fccfc6d502"
server_key_iv: "d197ea14d4a09e83e0c56c9e4c533e04"
server_key_version: "encryptionkey@0"
I20260809 05:36:16.403218 14090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "4007cb2cf31c45a597604e4b7ad55485"
format_stamp: "Formatted at 2026-08-09 05:36:16 on dist-test-slave-3kk6"
server_key: "47d10368592f1b13008c30fccfc6d502"
server_key_iv: "d197ea14d4a09e83e0c56c9e4c533e04"
server_key_version: "encryptionkey@0"
I20260809 05:36:16.410058 14090 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.006s
I20260809 05:36:16.414642 14115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:16.416306 14090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:36:16.416498 14090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "4007cb2cf31c45a597604e4b7ad55485"
format_stamp: "Formatted at 2026-08-09 05:36:16 on dist-test-slave-3kk6"
server_key: "47d10368592f1b13008c30fccfc6d502"
server_key_iv: "d197ea14d4a09e83e0c56c9e4c533e04"
server_key_version: "encryptionkey@0"
I20260809 05:36:16.416749 14090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:16.422605 14090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:16.423493 14090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:16.423775 14090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:16.424724 14090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:16.426201 14090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:16.426317 14090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:16.426455 14090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:16.426553 14090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:16.460237 14090 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:43507
I20260809 05:36:16.460320 14269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:43507 every 8 connection(s)
I20260809 05:36:16.461674 14090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:36:16.469390 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 14090
I20260809 05:36:16.469600 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:36:16.470077 31271 external_mini_cluster.cc:1428] Setting key 6dfb2942730531392aa61ad6e5ecff28
I20260809 05:36:16.470506 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:16.464344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:45989 (local address 127.30.137.254:33073)
0809 05:36:16.464598 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:16.464605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:16.466238 (+ 1633us) server_negotiation.cc:407] Connection header received
0809 05:36:16.468017 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:16.468022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:16.468093 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:16.468190 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:16.469689 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:16.469695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:16.469702 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:16.469725 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:16.469781 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:16.469807 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:16.469944 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:16.470142 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 05:36:16.470218 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 05:36:16.471657 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:16.465036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33073 (local address 127.30.137.194:45989)
0809 05:36:16.466028 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:16.466073 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:16.467757 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:16.468441 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:16.468449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:16.468539 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:16.469327 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:36:16.469329 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:16.469367 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:36:16.469454 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:16.469912 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:16.469923 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:16.469930 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 05:36:16.470229 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 05:36:16.470504 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":311,"threads_started":1}
I20260809 05:36:16.473146 14270 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33073
I20260809 05:36:16.473196 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33073
--builtin_ntp_servers=127.30.137.212:45033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 05:36:16.473516 14270 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:16.474244 14270 heartbeater.cc:511] Master 127.30.137.254:33073 requested a full tablet report, sending...
I20260809 05:36:16.475816 13828 ts_manager.cc:194] Registered new tserver with Master: 4007cb2cf31c45a597604e4b7ad55485 (127.30.137.194:43507)
W20260809 05:36:16.825052 14274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:16.825342 14274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:16.825498 14274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:16.833066 14274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:16.833180 14274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:16.833360 14274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:36:16.842630 14274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:33073
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:16.844089 14274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:16.845894 14274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:16.859716 14280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:16.859834 14283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:16.859754 14285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:16.860951 14274 server_base.cc:1034] running on GCE node
I20260809 05:36:16.861654 14274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:16.862695 14274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:16.863936 14274 hybrid_clock.cc:648] HybridClock initialized: now 1786253776863873 us; error 59 us; skew 500 ppm
I20260809 05:36:16.864291 14274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:36:16.866838 14274 webserver.cc:460] Webserver started at http://127.30.137.195:39703/ using document root <none> and password file <none>
I20260809 05:36:16.867564 14274 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:16.867704 14274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:16.868057 14274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:16.871160 14274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "9a0e1e6d263f4c04b68f226a8c135ac6"
format_stamp: "Formatted at 2026-08-09 05:36:16 on dist-test-slave-3kk6"
server_key: "e511fb3c4632f38b87820a9a7ad2f01c"
server_key_iv: "c4c7346cf3217b3c90ace0452e037a7c"
server_key_version: "encryptionkey@0"
I20260809 05:36:16.872129 14274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "9a0e1e6d263f4c04b68f226a8c135ac6"
format_stamp: "Formatted at 2026-08-09 05:36:16 on dist-test-slave-3kk6"
server_key: "e511fb3c4632f38b87820a9a7ad2f01c"
server_key_iv: "c4c7346cf3217b3c90ace0452e037a7c"
server_key_version: "encryptionkey@0"
I20260809 05:36:16.877576 14274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:36:16.881467 14293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:16.882905 14274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:36:16.883085 14274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "9a0e1e6d263f4c04b68f226a8c135ac6"
format_stamp: "Formatted at 2026-08-09 05:36:16 on dist-test-slave-3kk6"
server_key: "e511fb3c4632f38b87820a9a7ad2f01c"
server_key_iv: "c4c7346cf3217b3c90ace0452e037a7c"
server_key_version: "encryptionkey@0"
I20260809 05:36:16.883323 14274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:16.889381 14274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:16.890264 14274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:16.890574 14274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:16.891567 14274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:16.893203 14274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:16.893327 14274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:16.893469 14274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:16.893560 14274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:16.930053 14274 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40127
I20260809 05:36:16.930145 14462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40127 every 8 connection(s)
I20260809 05:36:16.931602 14274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:36:16.934501 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 14274
I20260809 05:36:16.934758 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:36:16.935321 31271 external_mini_cluster.cc:1428] Setting key cf3bd1166c18d9a1ada820b050f8da36
I20260809 05:36:16.941320 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:16.934527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:54431 (local address 127.30.137.254:33073)
0809 05:36:16.934751 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:16.934755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:16.936709 (+ 1954us) server_negotiation.cc:407] Connection header received
0809 05:36:16.939069 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:16.939074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:16.939159 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:16.939272 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:16.940606 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:16.940616 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:16.940621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:16.940640 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:16.940681 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:16.940703 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:16.940840 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:16.941038 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 05:36:16.941090 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:36:16.942293 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:16.935290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:33073 (local address 127.30.137.195:54431)
0809 05:36:16.936515 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:16.936576 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:16.938789 (+ 2213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:16.939530 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:16.939538 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:16.939632 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:16.940305 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:36:16.940308 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:16.940347 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:36:16.940435 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:16.940826 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:16.940836 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:16.940843 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 05:36:16.941079 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 05:36:16.941280 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":331,"threads_started":1}
I20260809 05:36:16.943852 14463 heartbeater.cc:347] Connected to a master server at 127.30.137.254:33073
I20260809 05:36:16.944159 14463 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:16.944743 14463 heartbeater.cc:511] Master 127.30.137.254:33073 requested a full tablet report, sending...
I20260809 05:36:16.946116 13828 ts_manager.cc:194] Registered new tserver with Master: 9a0e1e6d263f4c04b68f226a8c135ac6 (127.30.137.195:40127)
I20260809 05:36:16.951268 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 05:36:16.957425 14088 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:36:16.955836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.30.137.254:33073)
0809 05:36:16.956062 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:16.956071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:16.956512 (+ 441us) server_negotiation.cc:407] Connection header received
0809 05:36:16.956648 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:16.956652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:16.956705 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:16.956787 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:16.957181 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44796: BlockingRecv error: recv got EOF from 127.0.0.1:44796 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:36:16.957782 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 13905
I20260809 05:36:16.961663 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:16.958987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.30.137.193:39359)
0809 05:36:16.959570 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:16.959577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:16.959602 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:36:16.959660 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:16.959666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:16.959872 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:16.959995 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:16.960904 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:16.960911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:16.960921 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:16.960939 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:16.960978 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:16.960992 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:16.961096 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:16.961184 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 05:36:16.961306 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":322,"threads_started":1}
I20260809 05:36:16.962963 14076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:17.044523 14086 heartbeater.cc:503] Master 127.30.137.254:33073 was elected leader, sending a full tablet report...
I20260809 05:36:17.045774 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 13905
W20260809 05:36:17.068218 13808 connection.cc:535] server connection from 127.30.137.193:51287 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 05:36:17.068574 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 14090
I20260809 05:36:17.072726 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:17.069667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.30.137.194:43507)
0809 05:36:17.070251 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:17.070258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:17.070287 (+ 29us) server_negotiation.cc:407] Connection header received
0809 05:36:17.070368 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:17.070375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:17.070603 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:17.070819 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:17.071891 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:17.071897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:17.071910 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:17.071931 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:17.071973 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:17.071989 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:17.072108 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:17.072266 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 05:36:17.072392 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":294,"threads_started":1}
I20260809 05:36:17.074450 14264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:17.156994 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 14090
I20260809 05:36:17.184091 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 14274
I20260809 05:36:17.188608 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:17.185238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.30.137.195:40127)
0809 05:36:17.185887 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:17.185893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:17.185924 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:36:17.185990 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:17.185996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:17.186260 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:17.186415 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:17.187846 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:17.187854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:17.187866 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:36:17.187906 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:17.187972 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:36:17.187995 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:17.188155 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:17.188214 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 05:36:17.188300 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":327,"threads_started":1}
I20260809 05:36:17.189939 14455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:17.274837 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 14274
I20260809 05:36:17.296747 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 13786
I20260809 05:36:17.298017 13868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:17.403301 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 13786
2026-08-09T05:36:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2547 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:36:17 dist-test-slave-3kk6 krb5kdc[14488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:36:17 dist-test-slave-3kk6 krb5kdc[14488](info): set up 2 sockets
Aug 09 05:36:17 dist-test-slave-3kk6 krb5kdc[14488](info): commencing operation
krb5kdc: starting...
W20260809 05:36:18.739166 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:36:18 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:36:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:36:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:36:19.055927 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35131
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:36:19.414736 14510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:19.415150 14510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:19.415318 14510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:19.422923 14510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:36:19.423092 14510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:19.423199 14510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:19.423307 14510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:36:19.423384 14510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:36:19.433544 14510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:19.435057 14510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:19.436445 14510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:19.449182 14517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:19.449091 14520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:19.449144 14518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:19.450042 14510 server_base.cc:1034] running on GCE node
I20260809 05:36:19.450886 14510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:19.452348 14510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:19.453603 14510 hybrid_clock.cc:648] HybridClock initialized: now 1786253779453563 us; error 49 us; skew 500 ppm
I20260809 05:36:19.453982 14510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:36:19 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253779, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:36:19.457365 14510 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:36:19.459066 14510 webserver.cc:460] Webserver started at http://127.30.137.254:37047/ using document root <none> and password file <none>
I20260809 05:36:19.459794 14510 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:19.459952 14510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:19.460285 14510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:19.463570 14510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "d79f31a7e90b4fa58d854d5231a6f266"
format_stamp: "Formatted at 2026-08-09 05:36:19 on dist-test-slave-3kk6"
server_key: "729ebac71928d7ce288a1c2185384c41"
server_key_iv: "9a31ce2842e14aef985e6e3000dc4fa5"
server_key_version: "encryptionkey@0"
I20260809 05:36:19.464427 14510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "d79f31a7e90b4fa58d854d5231a6f266"
format_stamp: "Formatted at 2026-08-09 05:36:19 on dist-test-slave-3kk6"
server_key: "729ebac71928d7ce288a1c2185384c41"
server_key_iv: "9a31ce2842e14aef985e6e3000dc4fa5"
server_key_version: "encryptionkey@0"
I20260809 05:36:19.469782 14510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:36:19.473501 14530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:19.474850 14510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:36:19.475049 14510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "d79f31a7e90b4fa58d854d5231a6f266"
format_stamp: "Formatted at 2026-08-09 05:36:19 on dist-test-slave-3kk6"
server_key: "729ebac71928d7ce288a1c2185384c41"
server_key_iv: "9a31ce2842e14aef985e6e3000dc4fa5"
server_key_version: "encryptionkey@0"
I20260809 05:36:19.475265 14510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:19.491441 14510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:19.495688 14510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:19.496001 14510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:19.515214 14510 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35131
I20260809 05:36:19.515270 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35131 every 8 connection(s)
I20260809 05:36:19.516676 14510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:36:19.518352 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 14510
I20260809 05:36:19.518589 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:36:19.519317 31271 external_mini_cluster.cc:1428] Setting key 58b490ed3302fde402a0360baf12666b
I20260809 05:36:19.520913 14609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:36:19 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:36:19.534874 14609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266: Bootstrap starting.
I20260809 05:36:19.539218 14609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266: Neither blocks nor log segments found. Creating new log.
I20260809 05:36:19.540516 14609 log.cc:824] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266: Log is configured to *not* fsync() on all Append() calls
I20260809 05:36:19.544306 14609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266: No bootstrap required, opened a new log
I20260809 05:36:19.544649 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:19.521372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.30.137.254:35131)
0809 05:36:19.522251 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:19.522266 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:19.522329 (+ 63us) server_negotiation.cc:407] Connection header received
0809 05:36:19.524303 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:19.524331 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:19.524818 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:19.525239 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:19.526706 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:19.528766 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:19.530166 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:19.531345 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:19.531556 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:19.535659 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:19.535670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:19.535688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:19.535739 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:19.539916 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:19.540677 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:19.540686 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:19.540695 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:19.540909 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:19.541490 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:19.541494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:19.541496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:19.542085 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:19.542407 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:19.542844 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 05:36:19.543298 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":389,"threads_started":1}
I20260809 05:36:19.553023 14609 raft_consensus.cc:348] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79f31a7e90b4fa58d854d5231a6f266" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35131 } }
I20260809 05:36:19.553416 14609 raft_consensus.cc:374] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:36:19.553494 14609 raft_consensus.cc:729] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79f31a7e90b4fa58d854d5231a6f266, State: Initialized, Role: FOLLOWER
I20260809 05:36:19.554005 14609 consensus_queue.cc:260] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79f31a7e90b4fa58d854d5231a6f266" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35131 } }
I20260809 05:36:19.554221 14609 raft_consensus.cc:388] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:36:19.554328 14609 raft_consensus.cc:482] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:36:19.554463 14609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:36:19.556720 14609 raft_consensus.cc:504] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79f31a7e90b4fa58d854d5231a6f266" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35131 } }
I20260809 05:36:19.557188 14609 leader_election.cc:302] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d79f31a7e90b4fa58d854d5231a6f266; no voters:
I20260809 05:36:19.557730 14609 leader_election.cc:288] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:36:19.557891 14615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:36:19.559038 14615 raft_consensus.cc:686] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [term 1 LEADER]: Becoming Leader. State: Replica: d79f31a7e90b4fa58d854d5231a6f266, State: Running, Role: LEADER
I20260809 05:36:19.559659 14615 consensus_queue.cc:237] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79f31a7e90b4fa58d854d5231a6f266" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35131 } }
I20260809 05:36:19.560225 14609 sys_catalog.cc:564] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:36:19.562568 14617 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d79f31a7e90b4fa58d854d5231a6f266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79f31a7e90b4fa58d854d5231a6f266" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35131 } } }
I20260809 05:36:19.562585 14618 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d79f31a7e90b4fa58d854d5231a6f266. Latest consensus state: current_term: 1 leader_uuid: "d79f31a7e90b4fa58d854d5231a6f266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d79f31a7e90b4fa58d854d5231a6f266" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35131 } } }
I20260809 05:36:19.563205 14618 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [sys.catalog]: This master's current role is: LEADER
I20260809 05:36:19.563046 14617 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266 [sys.catalog]: This master's current role is: LEADER
I20260809 05:36:19.565151 14627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:36:19.571017 14627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:36:19.580693 14627 catalog_manager.cc:1347] Generated new cluster ID: c14f922a4ad54efa90542e187a87853a
I20260809 05:36:19.580842 14627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:36:19.604439 14627 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:36:19.605829 14627 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:36:19.611687 14627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266: Generated new TSK 0
I20260809 05:36:19.612805 14627 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:36:19.698405 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35131
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:36:20.057605 14649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:20.057977 14649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:20.058153 14649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:20.066263 14649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:20.066464 14649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:20.066689 14649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:36:20.077121 14649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:20.078557 14649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:20.080196 14649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:20.094391 14658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:20.094511 14662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:20.094442 14659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:20.095338 14649 server_base.cc:1034] running on GCE node
I20260809 05:36:20.099472 14649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:20.100989 14649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:20.102243 14649 hybrid_clock.cc:648] HybridClock initialized: now 1786253780102126 us; error 127 us; skew 500 ppm
I20260809 05:36:20.102604 14649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:36:20 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253780, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:36:20.106069 14649 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:36:20.107765 14649 webserver.cc:460] Webserver started at http://127.30.137.193:34965/ using document root <none> and password file <none>
I20260809 05:36:20.108525 14649 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:20.108675 14649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:20.109018 14649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:20.112335 14649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "83a9f18ba1714f0ab9adbe2c35e149fc"
format_stamp: "Formatted at 2026-08-09 05:36:20 on dist-test-slave-3kk6"
server_key: "bb24451e2fba1563b47a3565b10dd7e5"
server_key_iv: "66da52b7b4c23951cb06cf1d16df3b53"
server_key_version: "encryptionkey@0"
I20260809 05:36:20.113241 14649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "83a9f18ba1714f0ab9adbe2c35e149fc"
format_stamp: "Formatted at 2026-08-09 05:36:20 on dist-test-slave-3kk6"
server_key: "bb24451e2fba1563b47a3565b10dd7e5"
server_key_iv: "66da52b7b4c23951cb06cf1d16df3b53"
server_key_version: "encryptionkey@0"
I20260809 05:36:20.118854 14649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:36:20.122351 14670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:20.123821 14649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:36:20.124009 14649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "83a9f18ba1714f0ab9adbe2c35e149fc"
format_stamp: "Formatted at 2026-08-09 05:36:20 on dist-test-slave-3kk6"
server_key: "bb24451e2fba1563b47a3565b10dd7e5"
server_key_iv: "66da52b7b4c23951cb06cf1d16df3b53"
server_key_version: "encryptionkey@0"
I20260809 05:36:20.124214 14649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:20.153054 14649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:20.157390 14649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:20.157724 14649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:20.158804 14649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:20.160449 14649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:20.160561 14649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:20.160704 14649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:20.160791 14649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:20.196724 14649 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:33817
I20260809 05:36:20.196771 14838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:33817 every 8 connection(s)
I20260809 05:36:20.198148 14649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:36:20.201992 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 14649
I20260809 05:36:20.202247 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:36:20.203037 31271 external_mini_cluster.cc:1428] Setting key 910e6f3405903f499e501f4f9b27fdcf
Aug 09 05:36:20 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253780, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:36:20.219374 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:20.200994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:52667 (local address 127.30.137.254:35131)
0809 05:36:20.201463 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:20.201467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:20.202671 (+ 1204us) server_negotiation.cc:407] Connection header received
0809 05:36:20.204658 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:20.204664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:20.204767 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:20.204891 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:20.206467 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.207257 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.208858 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.209265 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.209439 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:20.213374 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:20.213380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:20.213383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:20.213418 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:20.216125 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:20.216938 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:20.216945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:20.216955 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:20.217084 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:20.217539 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:20.217545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:20.217548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:20.217823 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:20.217989 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:20.218736 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 05:36:20.218978 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":239,"threads_started":1}
I20260809 05:36:20.220381 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:20.201579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35131 (local address 127.30.137.193:52667)
0809 05:36:20.202446 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:20.202503 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:20.204361 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:20.205151 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:20.205160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:20.205696 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:20.206281 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:20.206298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.207426 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.207438 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:20.208694 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:20.208707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.209537 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.209547 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:20.209795 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:20.210561 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:20.210583 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:20.213127 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:20.216322 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:36:20.216331 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:36:20.216355 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:36:20.216785 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:36:20.217205 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:36:20.217210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:36:20.217214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:36:20.217415 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:36:20.217963 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:20.217971 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:20.218324 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 05:36:20.218600 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 05:36:20.219182 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":375,"threads_started":1}
I20260809 05:36:20.222072 14841 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35131
I20260809 05:36:20.222479 14841 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:20.223439 14841 heartbeater.cc:511] Master 127.30.137.254:35131 requested a full tablet report, sending...
I20260809 05:36:20.226025 14556 ts_manager.cc:194] Registered new tserver with Master: 83a9f18ba1714f0ab9adbe2c35e149fc (127.30.137.193:33817)
I20260809 05:36:20.228340 14556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:52667
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:36:20.273749 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35131
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:36:20.632589 14849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:20.632920 14849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:20.633023 14849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:20.640589 14849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:20.640722 14849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:20.640841 14849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:36:20.650753 14849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:20.652088 14849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:20.653435 14849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:20.667379 14857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:20.667443 14862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:20.667514 14859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:20.669003 14849 server_base.cc:1034] running on GCE node
I20260809 05:36:20.669661 14849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:20.670645 14849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:20.671958 14849 hybrid_clock.cc:648] HybridClock initialized: now 1786253780671901 us; error 46 us; skew 500 ppm
I20260809 05:36:20.672336 14849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:36:20 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253780, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:36:20.675818 14849 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:36:20.677466 14849 webserver.cc:460] Webserver started at http://127.30.137.194:33989/ using document root <none> and password file <none>
I20260809 05:36:20.678200 14849 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:20.679210 14849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:20.679665 14849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:20.684566 14849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "6167ed25fa8d42bca0c5a93df7ee430f"
format_stamp: "Formatted at 2026-08-09 05:36:20 on dist-test-slave-3kk6"
server_key: "d9e47579dc7e948a0ac5c8417ed9dca7"
server_key_iv: "378626229378b4d4986b346b9ac9010e"
server_key_version: "encryptionkey@0"
I20260809 05:36:20.685849 14849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "6167ed25fa8d42bca0c5a93df7ee430f"
format_stamp: "Formatted at 2026-08-09 05:36:20 on dist-test-slave-3kk6"
server_key: "d9e47579dc7e948a0ac5c8417ed9dca7"
server_key_iv: "378626229378b4d4986b346b9ac9010e"
server_key_version: "encryptionkey@0"
I20260809 05:36:20.694723 14849 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.005s
I20260809 05:36:20.699800 14872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:20.701987 14849 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 05:36:20.702203 14849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "6167ed25fa8d42bca0c5a93df7ee430f"
format_stamp: "Formatted at 2026-08-09 05:36:20 on dist-test-slave-3kk6"
server_key: "d9e47579dc7e948a0ac5c8417ed9dca7"
server_key_iv: "378626229378b4d4986b346b9ac9010e"
server_key_version: "encryptionkey@0"
I20260809 05:36:20.702495 14849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:20.749331 14849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:20.753729 14849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:20.753994 14849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:20.755020 14849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:20.756381 14849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:20.756464 14849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:20.756559 14849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:20.756604 14849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:20.790900 14849 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46493
I20260809 05:36:20.790938 15027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46493 every 8 connection(s)
I20260809 05:36:20.792274 14849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:36:20.799273 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 14849
I20260809 05:36:20.799497 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:36:20.799948 31271 external_mini_cluster.cc:1428] Setting key f3ce5f53f654bea020efe26b54f3f68d
Aug 09 05:36:20 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253780, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:36:20.813782 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:20.795121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:57053 (local address 127.30.137.254:35131)
0809 05:36:20.795590 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:20.795594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:20.796561 (+ 967us) server_negotiation.cc:407] Connection header received
0809 05:36:20.798377 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:20.798381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:20.798463 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:20.798563 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:20.800338 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.801307 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.803145 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.803595 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.803736 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:20.807492 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:20.807497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:20.807501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:20.807535 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:20.810417 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:20.811304 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:20.811309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:20.811311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:20.811456 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:20.811953 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:20.811957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:20.811959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:20.812235 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:20.812411 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:20.813141 (+ 730us) server_negotiation.cc:299] Negotiation successful
0809 05:36:20.813388 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":237,"threads_started":1}
I20260809 05:36:20.815132 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:20.795682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35131 (local address 127.30.137.194:57053)
0809 05:36:20.796390 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:20.796435 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:20.798102 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:20.798843 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:20.798851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:20.799436 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:20.800146 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:20.800172 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.801480 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.801493 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:20.802990 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:20.803004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:20.803853 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:20.803860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:20.804107 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:20.804877 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:20.804901 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:20.807226 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:20.810599 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:36:20.810608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:36:20.810630 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:36:20.811153 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:36:20.811582 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:36:20.811587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:36:20.811591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:36:20.811814 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:36:20.812376 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:20.812384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:20.812752 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 05:36:20.813039 (+ 287us) client_negotiation.cc:240] Negotiation successful
0809 05:36:20.813616 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":281,"threads_started":1}
I20260809 05:36:20.816757 15028 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35131
I20260809 05:36:20.817119 15028 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:20.818060 15028 heartbeater.cc:511] Master 127.30.137.254:35131 requested a full tablet report, sending...
I20260809 05:36:20.819836 14556 ts_manager.cc:194] Registered new tserver with Master: 6167ed25fa8d42bca0c5a93df7ee430f (127.30.137.194:46493)
I20260809 05:36:20.820693 14556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:57053
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:36:20.869817 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35131
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:36:21.232307 14841 heartbeater.cc:503] Master 127.30.137.254:35131 was elected leader, sending a full tablet report...
W20260809 05:36:21.235508 15039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:36:21.235803 15039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:36:21.236006 15039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:36:21.243443 15039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:36:21.243566 15039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:36:21.243773 15039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:36:21.253841 15039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:35131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:36:21.255235 15039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:36:21.256593 15039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:36:21.270370 15047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:21.270301 15048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:36:21.270332 15051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:36:21.271457 15039 server_base.cc:1034] running on GCE node
I20260809 05:36:21.272118 15039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:36:21.273012 15039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:36:21.274250 15039 hybrid_clock.cc:648] HybridClock initialized: now 1786253781274202 us; error 41 us; skew 500 ppm
I20260809 05:36:21.274580 15039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:36:21 dist-test-slave-3kk6 krb5kdc[14488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253781, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:36:21.281895 15039 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:36:21.283607 15039 webserver.cc:460] Webserver started at http://127.30.137.195:37685/ using document root <none> and password file <none>
I20260809 05:36:21.284402 15039 fs_manager.cc:359] Metadata directory not provided
I20260809 05:36:21.284564 15039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:36:21.284907 15039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:36:21.288126 15039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "f2ecd8697efb4be885262a434ce4fbcb"
format_stamp: "Formatted at 2026-08-09 05:36:21 on dist-test-slave-3kk6"
server_key: "9bd136c0ab609ffe9396e2b5e42715fc"
server_key_iv: "66aefbd50e90659eee247ca31c150110"
server_key_version: "encryptionkey@0"
I20260809 05:36:21.289011 15039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "f2ecd8697efb4be885262a434ce4fbcb"
format_stamp: "Formatted at 2026-08-09 05:36:21 on dist-test-slave-3kk6"
server_key: "9bd136c0ab609ffe9396e2b5e42715fc"
server_key_iv: "66aefbd50e90659eee247ca31c150110"
server_key_version: "encryptionkey@0"
I20260809 05:36:21.294543 15039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 05:36:21.298231 15062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:21.299726 15039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:36:21.299922 15039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "f2ecd8697efb4be885262a434ce4fbcb"
format_stamp: "Formatted at 2026-08-09 05:36:21 on dist-test-slave-3kk6"
server_key: "9bd136c0ab609ffe9396e2b5e42715fc"
server_key_iv: "66aefbd50e90659eee247ca31c150110"
server_key_version: "encryptionkey@0"
I20260809 05:36:21.300146 15039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:36:21.330596 15039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:36:21.337409 15039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:36:21.337801 15039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:36:21.339303 15039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:36:21.341466 15039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:36:21.341583 15039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:21.341738 15039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:36:21.341813 15039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:36:21.379879 15039 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:36303
I20260809 05:36:21.379904 15224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:36303 every 8 connection(s)
I20260809 05:36:21.382023 15039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:36:21.383499 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 15039
I20260809 05:36:21.383744 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:36:21.384307 31271 external_mini_cluster.cc:1428] Setting key b1fb1cea814ab5d4b9bcc89fce0d3fd6
Aug 09 05:36:21 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253781, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:36:21.407848 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.385784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:52729 (local address 127.30.137.254:35131)
0809 05:36:21.386365 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.386371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.387538 (+ 1167us) server_negotiation.cc:407] Connection header received
0809 05:36:21.389754 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.389763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.389885 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.390014 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:21.391637 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.393118 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.394779 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.395626 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.395785 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.400321 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:21.400327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:21.400330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:21.400371 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:21.403626 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.404771 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.404775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.404777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.404976 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.405557 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.405560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.405562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.405870 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:21.406038 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.407267 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.407512 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":262,"threads_started":1}
I20260809 05:36:21.409507 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.386473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35131 (local address 127.30.137.195:52729)
0809 05:36:21.387382 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:21.387433 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:21.389456 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:21.390238 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:21.390247 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:21.390872 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:36:21.391468 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.391487 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.393292 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.393296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.394583 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.394596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.395854 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.395859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.396078 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.396834 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:36:21.396873 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:36:21.399989 (+ 3116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:36:21.403826 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:36:21.403839 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:36:21.403862 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:36:21.404634 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:36:21.405095 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:36:21.405102 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:36:21.405109 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:36:21.405417 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:36:21.406020 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:36:21.406032 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:36:21.406639 (+ 607us) client_negotiation.cc:769] Sending connection context
0809 05:36:21.407061 (+ 422us) client_negotiation.cc:240] Negotiation successful
0809 05:36:21.407771 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":413,"threads_started":1}
I20260809 05:36:21.411329 15225 heartbeater.cc:347] Connected to a master server at 127.30.137.254:35131
I20260809 05:36:21.411691 15225 heartbeater.cc:464] Registering TS with master...
I20260809 05:36:21.412730 15225 heartbeater.cc:511] Master 127.30.137.254:35131 requested a full tablet report, sending...
I20260809 05:36:21.414434 14556 ts_manager.cc:194] Registered new tserver with Master: f2ecd8697efb4be885262a434ce4fbcb (127.30.137.195:36303)
I20260809 05:36:21.415351 14556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:52729
I20260809 05:36:21.426298 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:36:21.448316 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.431577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.30.137.254:35131)
0809 05:36:21.431835 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.431844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.432318 (+ 474us) server_negotiation.cc:407] Connection header received
0809 05:36:21.432488 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.432497 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.432632 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.432765 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:21.434255 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.435714 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.437011 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.437864 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.438021 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.440437 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:21.440447 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:21.440455 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:21.440505 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:21.444991 (+ 4486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.445750 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.445759 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.445764 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.445944 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.446324 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.446331 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.446336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.446914 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:21.447220 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.447543 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.447859 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 05:36:21.452948 14556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:36:21.460860 14556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:36:21.514767 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.476609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46493 (local address 127.0.0.1:54824)
0809 05:36:21.503038 (+ 26429us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:21.503115 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:21.503389 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:21.505704 (+ 2315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:21.505715 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:21.505755 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:36:21.506338 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.506373 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.508234 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.508241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.511176 (+ 2935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.511192 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.513552 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.513558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.513783 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.513844 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 05:36:21.513992 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 05:36:21.514231 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26306,"mutex_wait_us":7,"thread_start_us":27786,"threads_started":1}
I20260809 05:36:21.515808 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.476536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.30.137.194:46493)
0809 05:36:21.505127 (+ 28591us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.505134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.505178 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:36:21.505266 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.505272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.505481 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.505649 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:36:21.506550 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.508050 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.511386 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.513050 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.513217 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.513405 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.515277 (+ 1872us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.515461 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28413,"thread_start_us":28250,"threads_started":1}
I20260809 05:36:21.519516 14937 tablet_service.cc:1467] Processing CreateTablet for tablet 09900262d3024c83aaeae4578528d102 (DEFAULT_TABLE table=test-table [id=9bf1c8161e704e90a43b5013eb841196]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:36:21.520862 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.477320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.30.137.195:36303)
0809 05:36:21.501980 (+ 24660us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.501995 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.507556 (+ 5561us) server_negotiation.cc:407] Connection header received
0809 05:36:21.507785 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.507793 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.508104 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.508319 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:36:21.509535 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.511335 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.514049 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.515799 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.516094 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.516285 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.516924 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.520467 (+ 3543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24356,"thread_start_us":24691,"threads_started":1}
I20260809 05:36:21.521132 14937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09900262d3024c83aaeae4578528d102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:36:21.521920 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.488884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33817 (local address 127.0.0.1:60660)
0809 05:36:21.503866 (+ 14982us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:21.503888 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:21.504010 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:21.509252 (+ 5242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:21.509264 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:21.509361 (+ 97us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:36:21.510011 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.510027 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.513468 (+ 3441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.513476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.516419 (+ 2943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.516443 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.520444 (+ 4001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.520454 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.520729 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.520780 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 05:36:21.520862 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 05:36:21.521352 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14823,"mutex_wait_us":34,"thread_start_us":13588,"threads_started":1}
I20260809 05:36:21.522362 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.489178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.30.137.193:33817)
0809 05:36:21.508415 (+ 19237us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.508426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.508458 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:36:21.508561 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.508570 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.508807 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.511133 (+ 2326us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:36:21.511445 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.513263 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.516685 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.520075 (+ 3390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.521582 (+ 1507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.521762 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.521846 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.522018 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18987,"thread_start_us":19783,"threads_started":1}
I20260809 05:36:21.523164 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.476520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:36303 (local address 127.0.0.1:48380)
0809 05:36:21.503038 (+ 26518us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:21.507430 (+ 4392us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:21.507602 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:21.508664 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:21.508669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:21.508695 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:36:21.509287 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.509308 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.511610 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.511615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.513854 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.513867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.516513 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.516519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.516702 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.516739 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:36:21.521164 (+ 4425us) client_negotiation.cc:240] Negotiation successful
0809 05:36:21.521330 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26179,"mutex_wait_us":3,"thread_start_us":27559,"threads_started":1}
I20260809 05:36:21.526511 14743 tablet_service.cc:1467] Processing CreateTablet for tablet 09900262d3024c83aaeae4578528d102 (DEFAULT_TABLE table=test-table [id=9bf1c8161e704e90a43b5013eb841196]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:36:21.528102 14743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09900262d3024c83aaeae4578528d102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:36:21.527418 15135 tablet_service.cc:1467] Processing CreateTablet for tablet 09900262d3024c83aaeae4578528d102 (DEFAULT_TABLE table=test-table [id=9bf1c8161e704e90a43b5013eb841196]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:36:21.531096 15135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09900262d3024c83aaeae4578528d102. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:36:21.551698 15250 tablet_bootstrap.cc:492] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Bootstrap starting.
I20260809 05:36:21.554809 15253 tablet_bootstrap.cc:492] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Bootstrap starting.
I20260809 05:36:21.555970 15250 tablet_bootstrap.cc:654] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Neither blocks nor log segments found. Creating new log.
I20260809 05:36:21.557091 15250 log.cc:824] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Log is configured to *not* fsync() on all Append() calls
I20260809 05:36:21.559487 15253 tablet_bootstrap.cc:654] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Neither blocks nor log segments found. Creating new log.
I20260809 05:36:21.559509 15251 tablet_bootstrap.cc:492] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Bootstrap starting.
I20260809 05:36:21.560935 15253 log.cc:824] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Log is configured to *not* fsync() on all Append() calls
I20260809 05:36:21.563836 15250 tablet_bootstrap.cc:492] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: No bootstrap required, opened a new log
I20260809 05:36:21.564219 15250 ts_tablet_manager.cc:1397] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.002s
I20260809 05:36:21.564620 15251 tablet_bootstrap.cc:654] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Neither blocks nor log segments found. Creating new log.
I20260809 05:36:21.566390 15251 log.cc:824] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Log is configured to *not* fsync() on all Append() calls
I20260809 05:36:21.578198 15253 tablet_bootstrap.cc:492] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: No bootstrap required, opened a new log
I20260809 05:36:21.578197 15251 tablet_bootstrap.cc:492] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: No bootstrap required, opened a new log
I20260809 05:36:21.578540 15253 ts_tablet_manager.cc:1397] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Time spent bootstrapping tablet: real 0.024s user 0.014s sys 0.007s
I20260809 05:36:21.578540 15251 ts_tablet_manager.cc:1397] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260809 05:36:21.582135 15250 raft_consensus.cc:348] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.582909 15250 raft_consensus.cc:374] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:36:21.583084 15250 raft_consensus.cc:729] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6167ed25fa8d42bca0c5a93df7ee430f, State: Initialized, Role: FOLLOWER
I20260809 05:36:21.584103 15250 consensus_queue.cc:260] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.590160 15253 raft_consensus.cc:348] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.590729 15253 raft_consensus.cc:374] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:36:21.590845 15253 raft_consensus.cc:729] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ecd8697efb4be885262a434ce4fbcb, State: Initialized, Role: FOLLOWER
I20260809 05:36:21.591665 15253 consensus_queue.cc:260] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.591851 15251 raft_consensus.cc:348] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.592355 15251 raft_consensus.cc:374] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:36:21.592490 15251 raft_consensus.cc:729] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a9f18ba1714f0ab9adbe2c35e149fc, State: Initialized, Role: FOLLOWER
I20260809 05:36:21.593400 15251 consensus_queue.cc:260] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.595481 15028 heartbeater.cc:503] Master 127.30.137.254:35131 was elected leader, sending a full tablet report...
I20260809 05:36:21.596334 15250 ts_tablet_manager.cc:1428] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Time spent starting tablet: real 0.032s user 0.015s sys 0.015s
I20260809 05:36:21.597349 15225 heartbeater.cc:503] Master 127.30.137.254:35131 was elected leader, sending a full tablet report...
I20260809 05:36:21.598326 15253 ts_tablet_manager.cc:1428] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Time spent starting tablet: real 0.020s user 0.012s sys 0.008s
I20260809 05:36:21.600795 15251 ts_tablet_manager.cc:1428] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Time spent starting tablet: real 0.022s user 0.008s sys 0.012s
W20260809 05:36:21.634107 15228 tablet.cc:2369] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:36:21.674059 15259 raft_consensus.cc:482] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:36:21.674414 15259 raft_consensus.cc:504] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.676386 15259 leader_election.cc:288] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6167ed25fa8d42bca0c5a93df7ee430f (127.30.137.194:46493), 83a9f18ba1714f0ab9adbe2c35e149fc (127.30.137.193:33817)
I20260809 05:36:21.683795 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.677034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:40817 (local address 127.30.137.194:46493)
0809 05:36:21.677273 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.677278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.677609 (+ 331us) server_negotiation.cc:407] Connection header received
0809 05:36:21.677892 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.677897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.678003 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.678121 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:36:21.678721 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.679511 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.681701 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.682944 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.683064 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.683118 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.683492 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.683591 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 05:36:21.683902 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.676784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46493 (local address 127.30.137.195:40817)
0809 05:36:21.677467 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:21.677491 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:21.677677 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:21.678167 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:21.678171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:21.678194 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:36:21.678464 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.678478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.679672 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.679678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.681514 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.681533 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.683109 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.683115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.683308 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.683346 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:36:21.683462 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 05:36:21.683612 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 05:36:21.685087 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.677042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:60845 (local address 127.30.137.193:33817)
0809 05:36:21.677248 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.677253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.678311 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 05:36:21.678485 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.678490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.678601 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.678772 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:36:21.679322 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.680378 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.682765 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.684014 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.684182 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.684258 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.684701 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.684835 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:36:21.684950 14969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09900262d3024c83aaeae4578528d102" candidate_uuid: "f2ecd8697efb4be885262a434ce4fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" is_pre_election: true
I20260809 05:36:21.685150 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.677097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33817 (local address 127.30.137.195:60845)
0809 05:36:21.678177 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 05:36:21.678200 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:36:21.678346 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:36:21.678786 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:36:21.678791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:36:21.678815 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:36:21.679159 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.679169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.680572 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.680578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.682551 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:36:21.682568 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.684351 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.684361 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:36:21.684534 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.684557 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:36:21.684672 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 05:36:21.684812 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"thread_start_us":691,"threads_started":1}
I20260809 05:36:21.685528 14969 raft_consensus.cc:2454] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ecd8697efb4be885262a434ce4fbcb in term 0.
I20260809 05:36:21.686331 15068 leader_election.cc:302] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6167ed25fa8d42bca0c5a93df7ee430f, f2ecd8697efb4be885262a434ce4fbcb; no voters:
I20260809 05:36:21.686282 14771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09900262d3024c83aaeae4578528d102" candidate_uuid: "f2ecd8697efb4be885262a434ce4fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" is_pre_election: true
I20260809 05:36:21.686767 15259 raft_consensus.cc:2781] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:36:21.686920 15259 raft_consensus.cc:482] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:36:21.687019 15259 raft_consensus.cc:3037] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:36:21.686939 14771 raft_consensus.cc:2454] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ecd8697efb4be885262a434ce4fbcb in term 0.
I20260809 05:36:21.689453 15259 raft_consensus.cc:504] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
I20260809 05:36:21.690131 15259 leader_election.cc:288] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [CANDIDATE]: Term 1 election: Requested vote from peers 6167ed25fa8d42bca0c5a93df7ee430f (127.30.137.194:46493), 83a9f18ba1714f0ab9adbe2c35e149fc (127.30.137.193:33817)
I20260809 05:36:21.690606 14969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09900262d3024c83aaeae4578528d102" candidate_uuid: "f2ecd8697efb4be885262a434ce4fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6167ed25fa8d42bca0c5a93df7ee430f"
I20260809 05:36:21.690739 14771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09900262d3024c83aaeae4578528d102" candidate_uuid: "f2ecd8697efb4be885262a434ce4fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc"
I20260809 05:36:21.690893 14969 raft_consensus.cc:3037] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:36:21.690948 14771 raft_consensus.cc:3037] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:36:21.694348 14969 raft_consensus.cc:2454] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ecd8697efb4be885262a434ce4fbcb in term 1.
I20260809 05:36:21.694335 14771 raft_consensus.cc:2454] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ecd8697efb4be885262a434ce4fbcb in term 1.
I20260809 05:36:21.694859 15068 leader_election.cc:302] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6167ed25fa8d42bca0c5a93df7ee430f, f2ecd8697efb4be885262a434ce4fbcb; no voters:
I20260809 05:36:21.695176 15259 raft_consensus.cc:2781] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:36:21.695675 15259 raft_consensus.cc:686] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 1 LEADER]: Becoming Leader. State: Replica: f2ecd8697efb4be885262a434ce4fbcb, State: Running, Role: LEADER
I20260809 05:36:21.696180 15259 consensus_queue.cc:237] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } }
W20260809 05:36:21.701157 14842 tablet.cc:2369] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:36:21.701414 14554 catalog_manager.cc:5495] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb reported cstate change: term changed from 0 to 1, leader changed from <none> to f2ecd8697efb4be885262a434ce4fbcb (127.30.137.195). New cstate: current_term: 1 leader_uuid: "f2ecd8697efb4be885262a434ce4fbcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ecd8697efb4be885262a434ce4fbcb" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 } health_report { overall_health: UNKNOWN } } }
I20260809 05:36:21.740794 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.735782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.30.137.195:36303)
0809 05:36:21.735983 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.735987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.736005 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:36:21.736097 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.736101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.736190 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.736316 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:36:21.736671 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.737402 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.738749 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.739149 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.739244 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.739676 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:36:21.740193 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:36:21.740317 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.740417 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.740494 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 05:36:21.749189 14969 raft_consensus.cc:1261] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 1 FOLLOWER]: Refusing update from remote peer f2ecd8697efb4be885262a434ce4fbcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:36:21.749207 14771 raft_consensus.cc:1261] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 1 FOLLOWER]: Refusing update from remote peer f2ecd8697efb4be885262a434ce4fbcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:36:21.750272 15263 consensus_queue.cc:1035] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6167ed25fa8d42bca0c5a93df7ee430f" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:36:21.750623 15259 consensus_queue.cc:1035] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "83a9f18ba1714f0ab9adbe2c35e149fc" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:36:21.763818 15269 mvcc.cc:205] Tried to move back new op lower bound from 7316495490034028544 to 7316495489831636992. Current Snapshot: MvccSnapshot[applied={T|T < 7316495490034028544}]
I20260809 05:36:21.775785 14554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:36:21.776094 14554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:36:21.780059 14554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d79f31a7e90b4fa58d854d5231a6f266: Sending DeleteTablet for 3 replicas of tablet 09900262d3024c83aaeae4578528d102
I20260809 05:36:21.781361 14937 tablet_service.cc:1514] Processing DeleteTablet for tablet 09900262d3024c83aaeae4578528d102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:36:21 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54824
I20260809 05:36:21.781366 15135 tablet_service.cc:1514] Processing DeleteTablet for tablet 09900262d3024c83aaeae4578528d102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:36:21 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:48380
I20260809 05:36:21.781963 14743 tablet_service.cc:1514] Processing DeleteTablet for tablet 09900262d3024c83aaeae4578528d102 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:36:21 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:60660
I20260809 05:36:21.784313 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 14649
I20260809 05:36:21.792276 15283 tablet_replica.cc:330] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: stopping tablet replica
I20260809 05:36:21.792944 15283 raft_consensus.cc:2229] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:36:21.793650 15283 raft_consensus.cc:2258] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:36:21.797856 15281 tablet_replica.cc:330] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: stopping tablet replica
I20260809 05:36:21.798298 15281 raft_consensus.cc:2229] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:36:21.798988 15283 ts_tablet_manager.cc:1905] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:36:21.799458 15282 tablet_replica.cc:330] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: stopping tablet replica
I20260809 05:36:21.799937 15282 raft_consensus.cc:2229] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:36:21.800721 15282 raft_consensus.cc:2258] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:36:21.801084 15281 raft_consensus.cc:2258] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:36:21.804667 15282 ts_tablet_manager.cc:1905] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:36:21.805828 15281 ts_tablet_manager.cc:1905] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:36:21.806591 15283 ts_tablet_manager.cc:1918] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:36:21.806862 15283 log.cc:1196] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/09900262d3024c83aaeae4578528d102
I20260809 05:36:21.807478 15283 ts_tablet_manager.cc:1939] T 09900262d3024c83aaeae4578528d102 P 83a9f18ba1714f0ab9adbe2c35e149fc: Deleting consensus metadata
I20260809 05:36:21.809684 14532 catalog_manager.cc:4841] TS 83a9f18ba1714f0ab9adbe2c35e149fc (127.30.137.193:33817): tablet 09900262d3024c83aaeae4578528d102 (table test-table [id=9bf1c8161e704e90a43b5013eb841196]) successfully deleted
I20260809 05:36:21.815603 15282 ts_tablet_manager.cc:1918] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:36:21.815824 15282 log.cc:1196] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/09900262d3024c83aaeae4578528d102
I20260809 05:36:21.816453 15282 ts_tablet_manager.cc:1939] T 09900262d3024c83aaeae4578528d102 P f2ecd8697efb4be885262a434ce4fbcb: Deleting consensus metadata
Aug 09 05:36:21 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:36:21.818447 14531 catalog_manager.cc:4841] TS f2ecd8697efb4be885262a434ce4fbcb (127.30.137.195:36303): tablet 09900262d3024c83aaeae4578528d102 (table test-table [id=9bf1c8161e704e90a43b5013eb841196]) successfully deleted
I20260809 05:36:21.824821 15281 ts_tablet_manager.cc:1918] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:36:21.825023 15281 log.cc:1196] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/09900262d3024c83aaeae4578528d102
I20260809 05:36:21.825580 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.786344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.30.137.193:33817)
0809 05:36:21.786586 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.786596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.805251 (+ 18655us) server_negotiation.cc:407] Connection header received
0809 05:36:21.805822 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.805828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.805920 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.805996 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:21.807997 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.809170 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.811905 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.812555 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.812662 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.818477 (+ 5815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:21.818494 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:21.818512 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:21.818556 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:21.822430 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.823254 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.823262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.823264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.823405 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.823812 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.823817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.823821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.824192 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:21.824536 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.824847 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.825092 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:36:21.825620 15281 ts_tablet_manager.cc:1939] T 09900262d3024c83aaeae4578528d102 P 6167ed25fa8d42bca0c5a93df7ee430f: Deleting consensus metadata
I20260809 05:36:21.826267 14834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:21.827735 14536 catalog_manager.cc:4841] TS 6167ed25fa8d42bca0c5a93df7ee430f (127.30.137.194:46493): tablet 09900262d3024c83aaeae4578528d102 (table test-table [id=9bf1c8161e704e90a43b5013eb841196]) successfully deleted
I20260809 05:36:21.920148 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 14649
I20260809 05:36:21.956305 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 14849
Aug 09 05:36:21 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:36:21.974305 15244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:21.957167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.30.137.194:46493)
0809 05:36:21.957345 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:21.957349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:21.957371 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:36:21.957586 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:21.957591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:21.957655 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:21.957749 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:21.959453 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.960287 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.962230 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:21.962620 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:21.962806 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:21.968142 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:21.968159 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:21.968176 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:21.968212 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:21.971270 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.972039 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.972048 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.972050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.972180 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:21.972584 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:21.972590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:21.972598 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:21.973029 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:21.973360 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:21.973681 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 05:36:21.973937 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:36:21.975065 15021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:22.068152 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 14849
I20260809 05:36:22.095515 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 15039
Aug 09 05:36:22 dist-test-slave-3kk6 krb5kdc[14488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:36:22.117446 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:36:22.097107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.30.137.195:36303)
0809 05:36:22.097325 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 05:36:22.097332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:36:22.097362 (+ 30us) server_negotiation.cc:407] Connection header received
0809 05:36:22.097445 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:36:22.097450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:36:22.097564 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:36:22.097667 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:36:22.099971 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:22.101179 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:22.103344 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:36:22.104141 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:36:22.104312 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:36:22.110493 (+ 6181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:36:22.110508 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:36:22.110528 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:36:22.110571 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:36:22.114249 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:22.115133 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:22.115146 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:22.115150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:22.115306 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:36:22.115724 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:36:22.115730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:36:22.115737 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:36:22.116141 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:36:22.116522 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0809 05:36:22.116827 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 05:36:22.117077 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:36:22.118152 15218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:22.244172 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 15039
I20260809 05:36:22.273180 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 14510
I20260809 05:36:22.274475 14603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:36:22.387986 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 14510
2026-08-09T05:36:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4984 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:36:22 dist-test-slave-3kk6 krb5kdc[15292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:36:22 dist-test-slave-3kk6 krb5kdc[15292](info): set up 2 sockets
Aug 09 05:36:22 dist-test-slave-3kk6 krb5kdc[15292](info): commencing operation
krb5kdc: starting...
W20260809 05:36:23.732232 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:36:23 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:36:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/rangeradmin_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/rangeradmin_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/rangerlookup_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/rangerlookup_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
I20260809 05:36:24.179551 31271 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskUEwy2v/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/postgres -l logfile start
2026-08-09 05:36:26.749 UTC [15341] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 05:36:26.750 UTC [15341] LOG: listening on IPv4 address "127.30.137.212", port 49231
2026-08-09 05:36:26.753 UTC [15341] LOG: listening on Unix socket "/tmp/.s.PGSQL.49231"
2026-08-09 05:36:26.779 UTC [15343] LOG: database system was shut down at 2026-08-09 05:36:25 UTC
2026-08-09 05:36:26.783 UTC [15341] LOG: database system is ready to accept connections
I20260809 05:36:27.968200 31271 mini_postgres.cc:96] Postgres bound to 49231
2026-08-09 05:36:27.995 UTC [15354] FATAL: database "slave" does not exist
127.30.137.212:49231 - accepting connections
I20260809 05:36:27.996439 31271 mini_ranger.cc:161] Starting Ranger...
I20260809 05:36:28.031605 31271 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 05:36:28.518219 31271 mini_ranger.cc:87] Created ranger Postgres database
I20260809 05:36:28.518354 31271 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-admin
2026-08-09 05:36:28,871 [I] DB FLAVOR :POSTGRES
2026-08-09 05:36:28,872 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 05:36:28,872 [I] Checking connection..
2026-08-09 05:36:28,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 05:36:29,141 [I] Checking connection passed.
2026-08-09 05:36:29,141 [I] --------- Verifying version history table ---------
2026-08-09 05:36:29,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 05:36:29,404 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 05:36:29,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 05:36:29,684 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 05:36:29,684 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 05:36:29,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 05:36:29.954 UTC [15447] WARNING: there is no transaction in progress
2026-08-09 05:36:29,964 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 05:36:29,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 05:36:30,246 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 05:36:30,246 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 05:36:30,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 05:36:30,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 05:36:30,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-09 05:36:31,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 05:36:31,319 [I] Table x_portal_user does not exist in database ranger
2026-08-09 05:36:31,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 05:36:31,587 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 05:36:31,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 05:36:31,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 05:36:32,121 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 05:36:32,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 05:36:33.148 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.164 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.181 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.197 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.216 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.236 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.300 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.306 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.321 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.331 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.342 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.360 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.370 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.377 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.384 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:33.390 UTC [15602] WARNING: there is no transaction in progress
2026-08-09 05:36:34,020 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 05:36:34,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 05:36:34,272 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 05:36:34,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 05:36:34,539 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 05:36:34,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 05:36:34,816 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 05:36:34,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 05:36:35,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 05:36:35,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-09 05:36:35,556 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 05:36:35,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 05:36:35,829 [I] DB_PATCHES have already been applied
I20260809 05:36:35.835770 31271 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-admin:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskUEwy2v/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 05:36:35.835901 31271 mini_ranger.cc:193] Using host: 127.30.137.212
I20260809 05:36:35.860853 31271 mini_ranger.cc:227] Ranger admin URL: http://127.30.137.212:40607
Aug 09, 2026 5:36:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 5:36:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 5:36:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 5:36:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 5:36:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 5:36:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 5:36:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 05:36:37.287712 31271 mini_ranger.cc:160] Time spent starting Ranger: real 9.291s user 0.000s sys 0.077s
Aug 09, 2026 5:36:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 5:36:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 5:36:57 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 5:36:57 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 5:36:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 05:36:58.928722 31271 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/rangerkms_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/rangerkms_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.137.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.137.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 05:36:59 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253819, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 05:36:59.160871 31271 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 05:36:59.191169 31271 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 05:37:00.031623 31271 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 05:37:00.031831 31271 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms
I20260809 05:37:00.031985 31271 mini_ranger_kms.cc:227] Using postgres at 127.30.137.212:49231
2026-08-09 05:37:00,570 [I] DB FLAVOR :POSTGRES
2026-08-09 05:37:00,571 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 05:37:00,571 [I] Checking connection
2026-08-09 05:37:00,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 05:37:00,825 [I] connection success
2026-08-09 05:37:00,825 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 05:37:00,825 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 05:37:00,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 05:37:01,124 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 05:37:01,124 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 05:37:01,124 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 05:37:01,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:49231/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 05:37:01,421 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 05:37:01.925195 31271 mini_ranger_kms.cc:310] Created kms service
I20260809 05:37:02.115990 31271 mini_ranger_kms.cc:326] Created kudu user
I20260809 05:37:02.163187 31271 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 05:37:02 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253819, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:02.570580 31271 mini_ranger_kms.cc:384] Added ranger policy
I20260809 05:37:02.571067 31271 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms:/tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 05:37:02.571139 31271 mini_ranger_kms.cc:240] Using host: 127.30.137.212
I20260809 05:37:02.600025 31271 mini_ranger_kms.cc:276] Ranger KMS PID: 15859
I20260809 05:37:02.600270 31271 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.137.212:50551
Aug 09, 2026 5:37:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 5:37:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 5:37:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 5:37:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 5:37:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 5:37:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 05:37:05 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786253825, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.137.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:37:05.395720 31271 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.235s user 0.156s sys 0.423s
I20260809 05:37:05.396100 31271 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 05:37:05.396351 31271 mini_ranger_kms.cc:401] 127.30.137.212:50551/kms/v1/keys
Aug 09 05:37:05 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253825, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.137.212@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
Aug 09 05:37:06 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253825, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.137.212@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
Aug 09 05:37:08 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:37:08.525885 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44183
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:44183
--ranger_config_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:50551/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:37:08.918395 15931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:37:08.918820 15931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:37:08.918929 15931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:37:08.926592 15931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:37:08.926798 15931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:37:08.926857 15931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:37:08.926920 15931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:37:08.926970 15931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:37:08.937815 15931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.137.212:50551/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:44183
--ranger_config_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:44183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:37:08.939271 15931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:37:08.940621 15931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:37:08.953486 15939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:08.953423 15943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:08.953758 15938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:37:08.954720 15931 server_base.cc:1034] running on GCE node
I20260809 05:37:08.955677 15931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:37:08.957266 15931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:37:08.958503 15931 hybrid_clock.cc:648] HybridClock initialized: now 1786253828958468 us; error 38 us; skew 500 ppm
I20260809 05:37:08.958947 15931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:37:08 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:37:08.966833 15931 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:37:08.968585 15931 webserver.cc:460] Webserver started at http://127.30.137.254:39855/ using document root <none> and password file <none>
I20260809 05:37:08.969273 15931 fs_manager.cc:359] Metadata directory not provided
I20260809 05:37:08.969383 15931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:37:08.969666 15931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:37:08 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:09.032634 15931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "175cf89b853b477bac8348715d147a71"
format_stamp: "Formatted at 2026-08-09 05:37:09 on dist-test-slave-3kk6"
server_key: "e58250d30a26e7d341987752233d539b"
server_key_iv: "1fcb5b04f256e3f47dc77d62966c7163"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:09.033941 15931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "175cf89b853b477bac8348715d147a71"
format_stamp: "Formatted at 2026-08-09 05:37:09 on dist-test-slave-3kk6"
server_key: "e58250d30a26e7d341987752233d539b"
server_key_iv: "1fcb5b04f256e3f47dc77d62966c7163"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:09.041453 15931 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260809 05:37:09.062844 15955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:09.064682 15931 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260809 05:37:09.064920 15931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "175cf89b853b477bac8348715d147a71"
format_stamp: "Formatted at 2026-08-09 05:37:09 on dist-test-slave-3kk6"
server_key: "e58250d30a26e7d341987752233d539b"
server_key_iv: "1fcb5b04f256e3f47dc77d62966c7163"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:09.065204 15931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:37:09.095558 15931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:37:09.101203 15931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:37:09.101603 15931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:37:09.123657 15931 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:44183
I20260809 05:37:09.123682 16030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:44183 every 8 connection(s)
I20260809 05:37:09.125216 15931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:37:09.127871 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 15931
I20260809 05:37:09.128100 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:37:09.128947 31271 external_mini_cluster.cc:1428] Setting key cfa87af9200ccdf96bb25d78091779b1
I20260809 05:37:09.129971 16031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:37:09 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:37:09.145890 16031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71: Bootstrap starting.
I20260809 05:37:09.150434 16031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71: Neither blocks nor log segments found. Creating new log.
I20260809 05:37:09.151892 16031 log.cc:824] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71: Log is configured to *not* fsync() on all Append() calls
I20260809 05:37:09.155160 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:09.131234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.30.137.254:44183)
0809 05:37:09.132113 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:09.132125 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:09.132169 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:37:09.133781 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:09.133802 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:09.134214 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:09.134590 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:09.136348 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:09.137880 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:09.139735 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:09.140447 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:09.140593 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:09.145640 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:09.145648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:09.145665 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:09.145707 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:09.150032 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:09.150991 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:09.151003 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:09.151018 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:09.151235 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:09.151708 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:09.151713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:09.151716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:09.152517 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:09.152802 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:09.153311 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 05:37:09.153762 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":401,"threads_started":1}
I20260809 05:37:09.155689 16031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71: No bootstrap required, opened a new log
I20260809 05:37:09.163977 16031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cf89b853b477bac8348715d147a71" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44183 } }
I20260809 05:37:09.164363 16031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:37:09.164446 16031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175cf89b853b477bac8348715d147a71, State: Initialized, Role: FOLLOWER
I20260809 05:37:09.165024 16031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cf89b853b477bac8348715d147a71" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44183 } }
I20260809 05:37:09.165236 16031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:37:09.165342 16031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:37:09.165478 16031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:37:09.167738 16031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cf89b853b477bac8348715d147a71" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44183 } }
I20260809 05:37:09.168205 16031 leader_election.cc:302] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 175cf89b853b477bac8348715d147a71; no voters:
I20260809 05:37:09.168915 16031 leader_election.cc:288] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:37:09.169082 16037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:37:09.169905 16037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [term 1 LEADER]: Becoming Leader. State: Replica: 175cf89b853b477bac8348715d147a71, State: Running, Role: LEADER
I20260809 05:37:09.170675 16037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cf89b853b477bac8348715d147a71" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44183 } }
I20260809 05:37:09.171476 16031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:37:09.174285 16038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "175cf89b853b477bac8348715d147a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cf89b853b477bac8348715d147a71" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44183 } } }
I20260809 05:37:09.174772 16038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [sys.catalog]: This master's current role is: LEADER
I20260809 05:37:09.174256 16041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 175cf89b853b477bac8348715d147a71. Latest consensus state: current_term: 1 leader_uuid: "175cf89b853b477bac8348715d147a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175cf89b853b477bac8348715d147a71" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 44183 } } }
I20260809 05:37:09.175554 16041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71 [sys.catalog]: This master's current role is: LEADER
I20260809 05:37:09.182022 16047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:37:09.188220 16047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:37:09.188997 16031 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 05:37:09.209015 16047 catalog_manager.cc:1347] Generated new cluster ID: 2129d73941574bbc9b1687a915592104
I20260809 05:37:09.209225 16047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:37:09.251348 16047 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:37:09.253557 16047 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:37:09.271044 16047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71: Generated new TSK 0
I20260809 05:37:09.272454 16047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:37:10.055150 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:50551/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44183
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:37:10 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786253830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 05:37:10.565443 16093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:37:10.565912 16093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:37:10.566141 16093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:37:10.578423 16093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:37:10.578616 16093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:37:10.578883 16093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:37:10.595786 16093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.137.212:50551/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:37:10.597788 16093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:37:10.599684 16093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:37:10.619191 16107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:10.619174 16106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:37:10.627440 16093 server_base.cc:1034] running on GCE node
W20260809 05:37:10.628051 16111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:37:10.628940 16093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:37:10.636098 16093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:37:10.639580 16093 hybrid_clock.cc:648] HybridClock initialized: now 1786253830639493 us; error 85 us; skew 500 ppm
I20260809 05:37:10.640115 16093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:37:10 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:37:10.644340 16093 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:37:10.648947 16093 webserver.cc:460] Webserver started at http://127.30.137.193:40487/ using document root <none> and password file <none>
I20260809 05:37:10.649963 16093 fs_manager.cc:359] Metadata directory not provided
I20260809 05:37:10.650089 16093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:37:10.650496 16093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:37:10 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:10.671950 16093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "f80b43b164fc434d8c0a052b63367f1b"
format_stamp: "Formatted at 2026-08-09 05:37:10 on dist-test-slave-3kk6"
server_key: "83328b0657e098a38ab9a4dd04cdba32"
server_key_iv: "e13aec5cadfd0366b93a4158314b118e"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:10.673099 16093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "f80b43b164fc434d8c0a052b63367f1b"
format_stamp: "Formatted at 2026-08-09 05:37:10 on dist-test-slave-3kk6"
server_key: "83328b0657e098a38ab9a4dd04cdba32"
server_key_iv: "e13aec5cadfd0366b93a4158314b118e"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:10.680608 16093 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260809 05:37:10.701687 16120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:10.711006 16093 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.000s
I20260809 05:37:10.711390 16093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "f80b43b164fc434d8c0a052b63367f1b"
format_stamp: "Formatted at 2026-08-09 05:37:10 on dist-test-slave-3kk6"
server_key: "83328b0657e098a38ab9a4dd04cdba32"
server_key_iv: "e13aec5cadfd0366b93a4158314b118e"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:10.711814 16093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:37:10.775482 16093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:37:10.780985 16093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:37:10.781468 16093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:37:10.789062 16093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:37:10.791157 16093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:37:10.791280 16093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:10.791456 16093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:37:10.791589 16093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:10.851300 16093 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:35151
I20260809 05:37:10.851346 16287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:35151 every 8 connection(s)
I20260809 05:37:10.852905 16093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:37:10.862160 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 16093
I20260809 05:37:10.862383 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:37:10.863276 31271 external_mini_cluster.cc:1428] Setting key a918a12c7dcab289a0938ef72ee79018
Aug 09 05:37:10 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
Aug 09 05:37:10 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:37:10.930240 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:10.872026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:40239 (local address 127.30.137.254:44183)
0809 05:37:10.889360 (+ 17334us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:10.889371 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:10.890729 (+ 1358us) server_negotiation.cc:407] Connection header received
0809 05:37:10.893855 (+ 3126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:10.893864 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:10.893991 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:10.894079 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:10.898828 (+ 4749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:10.900043 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:10.907864 (+ 7821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:10.908575 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:10.908702 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:10.920906 (+ 12204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:10.920916 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:10.920921 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:10.920980 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:10.924949 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:10.926073 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:10.926079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:10.926083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:10.926262 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:10.926888 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:10.926893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:10.926897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:10.927268 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:10.927404 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:10.929559 (+ 2155us) server_negotiation.cc:299] Negotiation successful
0809 05:37:10.929836 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17123,"thread_start_us":17348,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
I20260809 05:37:10.933699 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:10.872910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44183 (local address 127.30.137.193:40239)
0809 05:37:10.890213 (+ 17303us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:10.890280 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:10.893352 (+ 3072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:10.894820 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:10.894835 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:10.895602 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:37:10.896548 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:10.896580 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:10.900309 (+ 3729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:10.900318 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:10.907552 (+ 7234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:10.907582 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:10.909137 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:10.909149 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:10.909509 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:10.914813 (+ 5304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:37:10.914863 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:37:10.920491 (+ 5628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:37:10.925173 (+ 4682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:37:10.925185 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:37:10.925207 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:37:10.925901 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:37:10.926428 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:37:10.926433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:37:10.926436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:37:10.926751 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:37:10.927503 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:37:10.927512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:37:10.928008 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 05:37:10.930580 (+ 2572us) client_negotiation.cc:240] Negotiation successful
0809 05:37:10.931257 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16949,"thread_start_us":16657,"threads_started":1}
I20260809 05:37:10.936313 16289 heartbeater.cc:347] Connected to a master server at 127.30.137.254:44183
I20260809 05:37:10.936725 16289 heartbeater.cc:464] Registering TS with master...
I20260809 05:37:10.937671 16289 heartbeater.cc:511] Master 127.30.137.254:44183 requested a full tablet report, sending...
I20260809 05:37:10.940389 15975 ts_manager.cc:194] Registered new tserver with Master: f80b43b164fc434d8c0a052b63367f1b (127.30.137.193:35151)
I20260809 05:37:10.942517 15975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:40239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
W20260809 05:37:10.986928 31271 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.30.137.194: real 0.120s user 0.007s sys 0.022s
I20260809 05:37:10.988775 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:50551/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44183
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:37:11 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
W20260809 05:37:11.517217 16295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:37:11.517583 16295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:37:11.517787 16295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:37:11.526518 16295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:37:11.526644 16295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:37:11.526859 16295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:37:11.537453 16295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.137.212:50551/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:37:11.538978 16295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:37:11.540447 16295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:37:11.555872 16315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:11.555882 16314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:11.557003 16318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:37:11.557368 16295 server_base.cc:1034] running on GCE node
I20260809 05:37:11.558060 16295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:37:11.559181 16295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:37:11.560423 16295 hybrid_clock.cc:648] HybridClock initialized: now 1786253831560359 us; error 60 us; skew 500 ppm
I20260809 05:37:11.560772 16295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:37:11 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:37:11.564405 16295 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:37:11.566154 16295 webserver.cc:460] Webserver started at http://127.30.137.194:38971/ using document root <none> and password file <none>
I20260809 05:37:11.566962 16295 fs_manager.cc:359] Metadata directory not provided
I20260809 05:37:11.567098 16295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:37:11.567441 16295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:37:11 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:11.586474 16295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "5facb48bf976414ea70b500a7acc3f4c"
format_stamp: "Formatted at 2026-08-09 05:37:11 on dist-test-slave-3kk6"
server_key: "95f7a41287c6f17c117b48294e82b624"
server_key_iv: "73d06279127cea95982ab03d1aa5b1c3"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:11.587751 16295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "5facb48bf976414ea70b500a7acc3f4c"
format_stamp: "Formatted at 2026-08-09 05:37:11 on dist-test-slave-3kk6"
server_key: "95f7a41287c6f17c117b48294e82b624"
server_key_iv: "73d06279127cea95982ab03d1aa5b1c3"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:11.595769 16295 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260809 05:37:11.616220 16328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:11.618300 16295 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s
I20260809 05:37:11.618613 16295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "5facb48bf976414ea70b500a7acc3f4c"
format_stamp: "Formatted at 2026-08-09 05:37:11 on dist-test-slave-3kk6"
server_key: "95f7a41287c6f17c117b48294e82b624"
server_key_iv: "73d06279127cea95982ab03d1aa5b1c3"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:11.619144 16295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:37:11.651422 16295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:37:11.657737 16295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:37:11.658190 16295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:37:11.659895 16295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:37:11.662125 16295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:37:11.662328 16295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 05:37:11.662514 16295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:37:11.662614 16295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:11.704969 16295 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:37283
I20260809 05:37:11.705031 16488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:37283 every 8 connection(s)
I20260809 05:37:11.706753 16295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:37:11.716446 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 16295
I20260809 05:37:11.716648 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:37:11.717263 31271 external_mini_cluster.cc:1428] Setting key bfdd8e38adecdb563b51620364a89c0e
Aug 09 05:37:11 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253831, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:37:11.764221 16492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:11.709799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:47219 (local address 127.30.137.254:44183)
0809 05:37:11.740321 (+ 30522us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:11.740329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:11.741791 (+ 1462us) server_negotiation.cc:407] Connection header received
0809 05:37:11.744541 (+ 2750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:11.744548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:11.744642 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:11.744773 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:11.750090 (+ 5317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:11.751125 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:11.752878 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:11.753310 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:11.753469 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:11.757722 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:11.757728 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:11.757731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:11.757770 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:11.760841 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:11.761654 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:11.761658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:11.761661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:11.761810 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:11.762282 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:11.762291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:11.762293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:11.762589 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:11.762788 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:11.763535 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 05:37:11.763798 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30346,"thread_start_us":30538,"threads_started":1}
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
I20260809 05:37:11.765363 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:11.710417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44183 (local address 127.30.137.194:47219)
0809 05:37:11.741108 (+ 30691us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:11.741189 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:11.744179 (+ 2990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:11.747113 (+ 2934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:11.747128 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:11.748864 (+ 1736us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:37:11.749795 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:11.749826 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:11.751327 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:11.751331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:11.752690 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:11.752703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:11.753572 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:11.753908 (+ 336us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:11.754152 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:11.754974 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:37:11.755000 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:37:11.757445 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:37:11.761010 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:37:11.761019 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:37:11.761039 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:37:11.761504 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:37:11.761932 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:37:11.761937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:37:11.761941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:37:11.762146 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:37:11.762766 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:37:11.762775 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:37:11.763135 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 05:37:11.763424 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 05:37:11.763989 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30341,"thread_start_us":30136,"threads_started":1}
I20260809 05:37:11.767284 16489 heartbeater.cc:347] Connected to a master server at 127.30.137.254:44183
I20260809 05:37:11.767629 16489 heartbeater.cc:464] Registering TS with master...
I20260809 05:37:11.768373 16489 heartbeater.cc:511] Master 127.30.137.254:44183 requested a full tablet report, sending...
I20260809 05:37:11.770471 15975 ts_manager.cc:194] Registered new tserver with Master: 5facb48bf976414ea70b500a7acc3f4c (127.30.137.194:37283)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
I20260809 05:37:11.771687 15975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:47219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:37:11.814020 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:50551/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44183
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:36237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:37:11.946498 16289 heartbeater.cc:503] Master 127.30.137.254:44183 was elected leader, sending a full tablet report...
W20260809 05:37:12.194314 16498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:37:12.194659 16498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:37:12.194869 16498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:37:12.202478 16498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:37:12.202644 16498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:37:12.202930 16498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:37:12.213105 16498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:36237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.137.212:50551/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:44183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:37:12.214450 16498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:37:12.215883 16498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:37:12.229924 16507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:12.229969 16509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:37:12.230017 16506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:37:12.236200 16498 server_base.cc:1034] running on GCE node
I20260809 05:37:12.236968 16498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:37:12.238063 16498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:37:12.239317 16498 hybrid_clock.cc:648] HybridClock initialized: now 1786253832239265 us; error 47 us; skew 500 ppm
I20260809 05:37:12.239655 16498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:37:12 dist-test-slave-3kk6 krb5kdc[15292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:37:12.243136 16498 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:37:12.244731 16498 webserver.cc:460] Webserver started at http://127.30.137.195:42059/ using document root <none> and password file <none>
I20260809 05:37:12.245469 16498 fs_manager.cc:359] Metadata directory not provided
I20260809 05:37:12.245608 16498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:37:12.245944 16498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:37:12 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:12.260790 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0"
format_stamp: "Formatted at 2026-08-09 05:37:12 on dist-test-slave-3kk6"
server_key: "b3638020b555bc62dbbee13fb4962fde"
server_key_iv: "d00d328d313e026bd89bddfd18968450"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:12.261713 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0"
format_stamp: "Formatted at 2026-08-09 05:37:12 on dist-test-slave-3kk6"
server_key: "b3638020b555bc62dbbee13fb4962fde"
server_key_iv: "d00d328d313e026bd89bddfd18968450"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:12.267928 16498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 05:37:12.284679 16519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:12.286494 16498 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260809 05:37:12.286723 16498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0"
format_stamp: "Formatted at 2026-08-09 05:37:12 on dist-test-slave-3kk6"
server_key: "b3638020b555bc62dbbee13fb4962fde"
server_key_iv: "d00d328d313e026bd89bddfd18968450"
server_key_version: "kuduclusterkey@0"
I20260809 05:37:12.287029 16498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:37:12.315471 16498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:37:12.319880 16498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:37:12.320256 16498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:37:12.321331 16498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:37:12.322892 16498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:37:12.322970 16498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:12.323153 16498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:37:12.323206 16498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:37:12.359628 16498 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:33043
I20260809 05:37:12.359674 16683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:33043 every 8 connection(s)
I20260809 05:37:12.361177 16498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:37:12.371251 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 16498
I20260809 05:37:12.371438 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:37:12.372057 31271 external_mini_cluster.cc:1428] Setting key 9949aa0a9f7f9648f194cb159ebc05f4
Aug 09 05:37:12 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:37:12.382047 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.363970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:50593 (local address 127.30.137.254:44183)
0809 05:37:12.364593 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.364597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.365361 (+ 764us) server_negotiation.cc:407] Connection header received
0809 05:37:12.367321 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.367327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.367428 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.367557 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:12.369083 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.369870 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.371931 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.372525 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.372673 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.376550 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:12.376555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:12.376558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:12.376591 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:12.379228 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.379943 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.379947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.379949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.380078 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.380491 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.380494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.380495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.380764 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:12.380874 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.381521 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.381778 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":319,"threads_started":1}
I20260809 05:37:12.383154 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.364421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:44183 (local address 127.30.137.195:50593)
0809 05:37:12.365168 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:12.365215 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:12.367024 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:12.367822 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:12.367830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:12.368305 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:37:12.368898 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.368917 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.370049 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.370053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.371748 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.371762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.372831 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.372842 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.373151 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.373939 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:37:12.373998 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:37:12.376333 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:37:12.379358 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:37:12.379365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:37:12.379379 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:37:12.379830 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:37:12.380174 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:37:12.380177 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:37:12.380179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:37:12.380389 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:37:12.380878 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:37:12.380884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:37:12.381206 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 05:37:12.381439 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 05:37:12.381947 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":311,"threads_started":1}
I20260809 05:37:12.384676 16685 heartbeater.cc:347] Connected to a master server at 127.30.137.254:44183
I20260809 05:37:12.384989 16685 heartbeater.cc:464] Registering TS with master...
I20260809 05:37:12.385694 16685 heartbeater.cc:511] Master 127.30.137.254:44183 requested a full tablet report, sending...
I20260809 05:37:12.387501 15975 ts_manager.cc:194] Registered new tserver with Master: ec5a51018a6a4d3fa9f254f8b588d1f0 (127.30.137.195:33043)
I20260809 05:37:12.388417 15975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:50593
I20260809 05:37:12.400640 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:37:12.419049 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.405711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.30.137.254:44183)
0809 05:37:12.405945 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.405955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.406424 (+ 469us) server_negotiation.cc:407] Connection header received
0809 05:37:12.406565 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.406569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.406712 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.406800 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:12.408406 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.409128 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.410457 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.410963 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.411062 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.413647 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:12.413652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:12.413655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:12.413687 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:12.416386 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.417112 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.417117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.417119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.417246 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.417596 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.417599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.417601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.417929 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:12.418130 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.418440 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.418616 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:37:12.423484 15975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:37:12.431067 15975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:37:12.470449 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.443511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:35151 (local address 127.0.0.1:48862)
0809 05:37:12.444601 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:12.444658 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:12.444849 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:12.462233 (+ 17384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:12.462240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:12.462276 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:37:12.463031 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.463053 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.464851 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.464856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.467080 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.467093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.469271 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.469278 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.469498 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.469558 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 05:37:12.469673 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 05:37:12.469917 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":45,"spinlock_wait_cycles":8064,"thread_start_us":583,"threads_started":1}
I20260809 05:37:12.470829 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.444628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.30.137.195:33043)
0809 05:37:12.454731 (+ 10103us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.454745 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.461063 (+ 6318us) server_negotiation.cc:407] Connection header received
0809 05:37:12.461396 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.461405 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.461718 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.461945 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:37:12.463702 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.465273 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.467961 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.469286 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.469482 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.469679 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.470226 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.470405 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9856,"thread_start_us":9661,"threads_started":1}
I20260809 05:37:12.470924 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.445083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37283 (local address 127.0.0.1:58390)
0809 05:37:12.461444 (+ 16361us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:12.461464 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:12.461594 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:12.462547 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:12.462553 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:12.462593 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:37:12.463093 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.463104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.464992 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.464998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.467710 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.467727 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.469925 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.469930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.470139 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.470173 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 05:37:12.470262 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 05:37:12.470414 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16228,"mutex_wait_us":210,"thread_start_us":16065,"threads_started":1}
I20260809 05:37:12.471252 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.445366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.30.137.194:37283)
0809 05:37:12.460760 (+ 15394us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.460769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.461755 (+ 986us) server_negotiation.cc:407] Connection header received
0809 05:37:12.461863 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.461871 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.462136 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.462357 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:37:12.463297 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.464810 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.467918 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.469376 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.469556 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.469740 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.470578 (+ 838us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.470795 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15187,"thread_start_us":15409,"threads_started":1}
I20260809 05:37:12.471426 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.444220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:33043 (local address 127.0.0.1:58048)
0809 05:37:12.460880 (+ 16660us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:12.460902 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:12.461114 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:12.462300 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:12.462304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:12.462324 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:37:12.463492 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.463508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.465455 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.465468 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.467796 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.467809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.469823 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.469827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.469960 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.470025 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 05:37:12.470135 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 05:37:12.470324 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16506,"mutex_wait_us":66,"thread_start_us":16342,"threads_started":1}
I20260809 05:37:12.472276 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.443864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.30.137.193:35151)
0809 05:37:12.461574 (+ 17710us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.461583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.461611 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:37:12.461720 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.461729 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.461965 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.462764 (+ 799us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:37:12.463219 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.464645 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.467261 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.468655 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.471416 (+ 2761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.471613 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.471703 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.471896 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17526,"thread_start_us":18690,"threads_started":1}
I20260809 05:37:12.475875 16588 tablet_service.cc:1467] Processing CreateTablet for tablet e1b46ccf9a1449e6aafe55f96611e7e9 (DEFAULT_TABLE table=test-table [id=dec453e4196a46188f467098d099fe4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:37:12.475878 16404 tablet_service.cc:1467] Processing CreateTablet for tablet e1b46ccf9a1449e6aafe55f96611e7e9 (DEFAULT_TABLE table=test-table [id=dec453e4196a46188f467098d099fe4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:37:12.477560 16404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1b46ccf9a1449e6aafe55f96611e7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:37:12.477566 16588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1b46ccf9a1449e6aafe55f96611e7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:37:12.480114 16198 tablet_service.cc:1467] Processing CreateTablet for tablet e1b46ccf9a1449e6aafe55f96611e7e9 (DEFAULT_TABLE table=test-table [id=dec453e4196a46188f467098d099fe4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:37:12.481510 16198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1b46ccf9a1449e6aafe55f96611e7e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:37:12.494272 16713 tablet_bootstrap.cc:492] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Bootstrap starting.
I20260809 05:37:12.496896 16715 tablet_bootstrap.cc:492] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Bootstrap starting.
I20260809 05:37:12.498348 16712 tablet_bootstrap.cc:492] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Bootstrap starting.
I20260809 05:37:12.499022 16713 tablet_bootstrap.cc:654] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Neither blocks nor log segments found. Creating new log.
I20260809 05:37:12.500504 16713 log.cc:824] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:37:12.500890 16715 tablet_bootstrap.cc:654] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Neither blocks nor log segments found. Creating new log.
I20260809 05:37:12.502154 16712 tablet_bootstrap.cc:654] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Neither blocks nor log segments found. Creating new log.
I20260809 05:37:12.502257 16715 log.cc:824] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:37:12.503587 16712 log.cc:824] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Log is configured to *not* fsync() on all Append() calls
I20260809 05:37:12.505445 16713 tablet_bootstrap.cc:492] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: No bootstrap required, opened a new log
I20260809 05:37:12.505733 16713 ts_tablet_manager.cc:1397] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260809 05:37:12.506245 16715 tablet_bootstrap.cc:492] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: No bootstrap required, opened a new log
I20260809 05:37:12.506510 16715 ts_tablet_manager.cc:1397] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 05:37:12.506755 16712 tablet_bootstrap.cc:492] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: No bootstrap required, opened a new log
I20260809 05:37:12.507014 16712 ts_tablet_manager.cc:1397] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 05:37:12.514072 16715 raft_consensus.cc:348] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.514487 16715 raft_consensus.cc:374] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:37:12.514621 16715 raft_consensus.cc:729] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80b43b164fc434d8c0a052b63367f1b, State: Initialized, Role: FOLLOWER
I20260809 05:37:12.515323 16715 consensus_queue.cc:260] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.516960 16715 ts_tablet_manager.cc:1428] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 05:37:12.517544 16713 raft_consensus.cc:348] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.517606 16712 raft_consensus.cc:348] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.518029 16712 raft_consensus.cc:374] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:37:12.518011 16713 raft_consensus.cc:374] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:37:12.518146 16712 raft_consensus.cc:729] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5a51018a6a4d3fa9f254f8b588d1f0, State: Initialized, Role: FOLLOWER
I20260809 05:37:12.518146 16713 raft_consensus.cc:729] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5facb48bf976414ea70b500a7acc3f4c, State: Initialized, Role: FOLLOWER
I20260809 05:37:12.518818 16712 consensus_queue.cc:260] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.518908 16713 consensus_queue.cc:260] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.519905 16685 heartbeater.cc:503] Master 127.30.137.254:44183 was elected leader, sending a full tablet report...
I20260809 05:37:12.520608 16712 ts_tablet_manager.cc:1428] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260809 05:37:12.522567 16713 ts_tablet_manager.cc:1428] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260809 05:37:12.523144 16489 heartbeater.cc:503] Master 127.30.137.254:44183 was elected leader, sending a full tablet report...
I20260809 05:37:12.595918 16720 raft_consensus.cc:482] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:37:12.596289 16720 raft_consensus.cc:504] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.598080 16720 leader_election.cc:288] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5facb48bf976414ea70b500a7acc3f4c (127.30.137.194:37283), ec5a51018a6a4d3fa9f254f8b588d1f0 (127.30.137.195:33043)
I20260809 05:37:12.605438 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.598793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:36853 (local address 127.30.137.194:37283)
0809 05:37:12.599018 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.599026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.599735 (+ 709us) server_negotiation.cc:407] Connection header received
0809 05:37:12.599939 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.599943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.600005 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.600104 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:37:12.600711 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.601442 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.603645 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.604425 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.604604 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.604684 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.605024 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.605152 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:37:12.605551 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.598587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:49391 (local address 127.30.137.195:33043)
0809 05:37:12.598862 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.598869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.599855 (+ 986us) server_negotiation.cc:407] Connection header received
0809 05:37:12.600077 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.600081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.600142 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.600253 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:37:12.600938 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.601644 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.603761 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.604572 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.604765 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.604849 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.605156 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.605294 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260809 05:37:12.605650 16726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.598691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37283 (local address 127.30.137.193:36853)
0809 05:37:12.599604 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:12.599621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:12.599766 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:12.600149 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:12.600155 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:12.600182 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:37:12.600555 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.600566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.601577 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.601584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.603479 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.603493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.604650 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.604656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.604843 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.604881 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 05:37:12.604996 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 05:37:12.605130 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":4,"thread_start_us":525,"threads_started":1}
I20260809 05:37:12.605877 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.598735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:33043 (local address 127.30.137.193:49391)
0809 05:37:12.599734 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0809 05:37:12.599754 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:37:12.599917 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:37:12.600273 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:37:12.600277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:37:12.600320 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:37:12.600767 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.600776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.601801 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.601806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.603599 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:37:12.603612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.604822 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.604828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:37:12.604992 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.605015 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:37:12.605126 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 05:37:12.605250 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":499,"threads_started":1}
I20260809 05:37:12.606832 16431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b46ccf9a1449e6aafe55f96611e7e9" candidate_uuid: "f80b43b164fc434d8c0a052b63367f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5facb48bf976414ea70b500a7acc3f4c" is_pre_election: true
I20260809 05:37:12.606911 16618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b46ccf9a1449e6aafe55f96611e7e9" candidate_uuid: "f80b43b164fc434d8c0a052b63367f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" is_pre_election: true
I20260809 05:37:12.607378 16431 raft_consensus.cc:2454] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80b43b164fc434d8c0a052b63367f1b in term 0.
I20260809 05:37:12.607427 16618 raft_consensus.cc:2454] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f80b43b164fc434d8c0a052b63367f1b in term 0.
I20260809 05:37:12.608107 16124 leader_election.cc:302] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5facb48bf976414ea70b500a7acc3f4c, f80b43b164fc434d8c0a052b63367f1b; no voters:
I20260809 05:37:12.608537 16720 raft_consensus.cc:2781] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:37:12.608695 16720 raft_consensus.cc:482] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:37:12.608820 16720 raft_consensus.cc:3037] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:37:12.611280 16720 raft_consensus.cc:504] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
W20260809 05:37:12.612183 16290 tablet.cc:2369] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:37:12.612344 16431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b46ccf9a1449e6aafe55f96611e7e9" candidate_uuid: "f80b43b164fc434d8c0a052b63367f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5facb48bf976414ea70b500a7acc3f4c"
I20260809 05:37:12.612493 16618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b46ccf9a1449e6aafe55f96611e7e9" candidate_uuid: "f80b43b164fc434d8c0a052b63367f1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0"
I20260809 05:37:12.612558 16431 raft_consensus.cc:3037] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:37:12.612713 16618 raft_consensus.cc:3037] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 0 FOLLOWER]: Advancing to term 1
W20260809 05:37:12.613278 16687 tablet.cc:2369] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:37:12.614014 16720 leader_election.cc:288] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [CANDIDATE]: Term 1 election: Requested vote from peers 5facb48bf976414ea70b500a7acc3f4c (127.30.137.194:37283), ec5a51018a6a4d3fa9f254f8b588d1f0 (127.30.137.195:33043)
I20260809 05:37:12.615204 16431 raft_consensus.cc:2454] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80b43b164fc434d8c0a052b63367f1b in term 1.
I20260809 05:37:12.615556 16618 raft_consensus.cc:2454] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f80b43b164fc434d8c0a052b63367f1b in term 1.
I20260809 05:37:12.615667 16124 leader_election.cc:302] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5facb48bf976414ea70b500a7acc3f4c, f80b43b164fc434d8c0a052b63367f1b; no voters:
I20260809 05:37:12.616012 16720 raft_consensus.cc:2781] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:37:12.616521 16720 raft_consensus.cc:686] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 1 LEADER]: Becoming Leader. State: Replica: f80b43b164fc434d8c0a052b63367f1b, State: Running, Role: LEADER
I20260809 05:37:12.617050 16720 consensus_queue.cc:237] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } }
I20260809 05:37:12.621814 15972 catalog_manager.cc:5495] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b reported cstate change: term changed from 0 to 1, leader changed from <none> to f80b43b164fc434d8c0a052b63367f1b (127.30.137.193). New cstate: current_term: 1 leader_uuid: "f80b43b164fc434d8c0a052b63367f1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80b43b164fc434d8c0a052b63367f1b" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 35151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 } health_report { overall_health: UNKNOWN } } }
I20260809 05:37:12.667613 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.662124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.30.137.193:35151)
0809 05:37:12.662359 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.662364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.662383 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:37:12.662569 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.662572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.662678 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.662808 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:37:12.663168 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.663939 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.665468 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.665875 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.665967 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.666531 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:37:12.667089 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:37:12.667201 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.667310 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.667406 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 05:37:12.676357 16618 raft_consensus.cc:1261] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 1 FOLLOWER]: Refusing update from remote peer f80b43b164fc434d8c0a052b63367f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:37:12.676393 16431 raft_consensus.cc:1261] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 1 FOLLOWER]: Refusing update from remote peer f80b43b164fc434d8c0a052b63367f1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:37:12.677373 16729 consensus_queue.cc:1035] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec5a51018a6a4d3fa9f254f8b588d1f0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 33043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:37:12.677736 16720 consensus_queue.cc:1035] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5facb48bf976414ea70b500a7acc3f4c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:37:12.689898 16734 mvcc.cc:205] Tried to move back new op lower bound from 7316495698632077312 to 7316495698403897344. Current Snapshot: MvccSnapshot[applied={T|T < 7316495698632077312}]
I20260809 05:37:12.706576 15972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:37:12.706941 15972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:37:12.711238 15972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 175cf89b853b477bac8348715d147a71: Sending DeleteTablet for 3 replicas of tablet e1b46ccf9a1449e6aafe55f96611e7e9
I20260809 05:37:12.712882 16588 tablet_service.cc:1514] Processing DeleteTablet for tablet e1b46ccf9a1449e6aafe55f96611e7e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:37:12 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:58048
I20260809 05:37:12.712893 16404 tablet_service.cc:1514] Processing DeleteTablet for tablet e1b46ccf9a1449e6aafe55f96611e7e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:37:12 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:58390
I20260809 05:37:12.713639 16198 tablet_service.cc:1514] Processing DeleteTablet for tablet e1b46ccf9a1449e6aafe55f96611e7e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:37:12 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:48862
I20260809 05:37:12.724193 16751 tablet_replica.cc:330] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: stopping tablet replica
I20260809 05:37:12.724865 16750 tablet_replica.cc:330] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: stopping tablet replica
I20260809 05:37:12.725682 16750 raft_consensus.cc:2229] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:37:12.726078 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 16093
I20260809 05:37:12.726547 16751 raft_consensus.cc:2229] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:37:12.727520 16751 raft_consensus.cc:2258] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:37:12.728132 16750 raft_consensus.cc:2258] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:37:12.729128 16749 tablet_replica.cc:330] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: stopping tablet replica
I20260809 05:37:12.731038 16749 raft_consensus.cc:2229] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:37:12.732580 16750 ts_tablet_manager.cc:1905] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:37:12.733171 16749 raft_consensus.cc:2258] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:37:12.733538 16751 ts_tablet_manager.cc:1905] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:37:12.736277 16749 ts_tablet_manager.cc:1905] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:37:12.740960 16750 ts_tablet_manager.cc:1918] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:37:12.741205 16750 log.cc:1196] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/e1b46ccf9a1449e6aafe55f96611e7e9
I20260809 05:37:12.741299 16751 ts_tablet_manager.cc:1918] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:37:12.741489 16751 log.cc:1196] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/e1b46ccf9a1449e6aafe55f96611e7e9
I20260809 05:37:12.741981 16750 ts_tablet_manager.cc:1939] T e1b46ccf9a1449e6aafe55f96611e7e9 P 5facb48bf976414ea70b500a7acc3f4c: Deleting consensus metadata
I20260809 05:37:12.742062 16751 ts_tablet_manager.cc:1939] T e1b46ccf9a1449e6aafe55f96611e7e9 P f80b43b164fc434d8c0a052b63367f1b: Deleting consensus metadata
I20260809 05:37:12.744215 15959 catalog_manager.cc:4841] TS 5facb48bf976414ea70b500a7acc3f4c (127.30.137.194:37283): tablet e1b46ccf9a1449e6aafe55f96611e7e9 (table test-table [id=dec453e4196a46188f467098d099fe4e]) successfully deleted
I20260809 05:37:12.744589 15959 catalog_manager.cc:4841] TS f80b43b164fc434d8c0a052b63367f1b (127.30.137.193:35151): tablet e1b46ccf9a1449e6aafe55f96611e7e9 (table test-table [id=dec453e4196a46188f467098d099fe4e]) successfully deleted
I20260809 05:37:12.746130 16749 ts_tablet_manager.cc:1918] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:37:12.746313 16749 log.cc:1196] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/e1b46ccf9a1449e6aafe55f96611e7e9
I20260809 05:37:12.746901 16749 ts_tablet_manager.cc:1939] T e1b46ccf9a1449e6aafe55f96611e7e9 P ec5a51018a6a4d3fa9f254f8b588d1f0: Deleting consensus metadata
I20260809 05:37:12.748874 15956 catalog_manager.cc:4841] TS ec5a51018a6a4d3fa9f254f8b588d1f0 (127.30.137.195:33043): tablet e1b46ccf9a1449e6aafe55f96611e7e9 (table test-table [id=dec453e4196a46188f467098d099fe4e]) successfully deleted
Aug 09 05:37:12 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:37:12.759268 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.727529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.30.137.193:35151)
0809 05:37:12.727722 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.727728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.731863 (+ 4135us) server_negotiation.cc:407] Connection header received
0809 05:37:12.732192 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.732198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.732280 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.732361 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:12.743413 (+ 11052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.744463 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.745968 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.746697 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.746857 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.751852 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:12.751870 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:12.751895 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:12.751967 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:12.756182 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.756893 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.756904 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.756908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.757089 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.757485 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.757492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.757500 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.757987 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:12.758351 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.758533 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.758884 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:37:12.760146 16281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:37:12.861325 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 16093
I20260809 05:37:12.884647 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 16295
Aug 09 05:37:12 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:37:12.900821 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:12.885551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.30.137.194:37283)
0809 05:37:12.885738 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:12.885742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:12.885783 (+ 41us) server_negotiation.cc:407] Connection header received
0809 05:37:12.885865 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:12.885874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:12.885937 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:12.886033 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:12.887436 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.888232 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.889578 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:12.889971 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:12.890066 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:12.894818 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:12.894834 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:12.894851 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:12.894885 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:12.897827 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.898572 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.898581 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.898583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.898853 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:12.899355 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:12.899359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:12.899361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:12.899805 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:12.900063 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:12.900369 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 05:37:12.900541 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:37:12.901723 16483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:37:12.991781 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 16295
I20260809 05:37:13.014251 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 16498
Aug 09 05:37:13 dist-test-slave-3kk6 krb5kdc[15292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:37:13.029563 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:37:13.015168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.30.137.195:33043)
0809 05:37:13.015349 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 05:37:13.015354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:37:13.015373 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:37:13.015513 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:37:13.015517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:37:13.015596 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:37:13.015679 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:37:13.017038 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:13.017767 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:13.019160 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:37:13.019542 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:37:13.019636 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:37:13.024059 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:37:13.024076 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:37:13.024091 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:37:13.024124 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:37:13.027064 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:13.027703 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:13.027711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:13.027713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:13.027852 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:37:13.028170 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:37:13.028173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:37:13.028175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:37:13.028529 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:37:13.028748 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 05:37:13.029004 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 05:37:13.029175 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:37:13.030233 16677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:37:13.120313 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 16498
I20260809 05:37:13.149677 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 15931
I20260809 05:37:13.150956 16023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:37:13.265933 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 15931
I20260809 05:37:13.304788 31271 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 05:37:13.650910 31271 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 05:37:13.651232 31271 mini_ranger.cc:66] Stopping Ranger...
I20260809 05:37:14.006989 31271 mini_ranger.cc:68] Stopped Ranger
2026-08-09 05:37:14.007 UTC [15341] LOG: received smart shutdown request
2026-08-09 05:37:14.011 UTC [15341] LOG: background worker "logical replication launcher" (PID 15350) exited with exit code 1
2026-08-09 05:37:14.012 UTC [15345] LOG: shutting down
2026-08-09 05:37:14.183 UTC [15341] LOG: database system is shut down
2026-08-09T05:37:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51943 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:37:14 dist-test-slave-3kk6 krb5kdc[16773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:37:14 dist-test-slave-3kk6 krb5kdc[16773](info): set up 2 sockets
Aug 09 05:37:14 dist-test-slave-3kk6 krb5kdc[16773](info): commencing operation
krb5kdc: starting...
W20260809 05:37:15.657483 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:37:15 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:37:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:37:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/rangeradmin_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/rangeradmin_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/rangerlookup_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/rangerlookup_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
I20260809 05:37:16.073935 31271 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskUEwy2v/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/postgres -l logfile start
2026-08-09 05:37:18.556 UTC [16819] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 05:37:18.556 UTC [16819] LOG: listening on IPv4 address "127.30.137.212", port 55101
2026-08-09 05:37:18.559 UTC [16819] LOG: listening on Unix socket "/tmp/.s.PGSQL.55101"
2026-08-09 05:37:18.584 UTC [16823] LOG: database system was shut down at 2026-08-09 05:37:17 UTC
2026-08-09 05:37:18.588 UTC [16819] LOG: database system is ready to accept connections
I20260809 05:37:19.775856 31271 mini_postgres.cc:96] Postgres bound to 55101
2026-08-09 05:37:19.801 UTC [16835] FATAL: database "slave" does not exist
127.30.137.212:55101 - accepting connections
I20260809 05:37:19.802421 31271 mini_ranger.cc:161] Starting Ranger...
I20260809 05:37:19.836839 31271 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 05:37:20.311789 31271 mini_ranger.cc:87] Created ranger Postgres database
I20260809 05:37:20.311931 31271 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-admin
2026-08-09 05:37:20,626 [I] DB FLAVOR :POSTGRES
2026-08-09 05:37:20,627 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 05:37:20,627 [I] Checking connection..
2026-08-09 05:37:20,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 05:37:20,896 [I] Checking connection passed.
2026-08-09 05:37:20,896 [I] --------- Verifying version history table ---------
2026-08-09 05:37:20,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 05:37:21,178 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 05:37:21,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 05:37:21,449 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 05:37:21,449 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 05:37:21,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 05:37:21.714 UTC [16932] WARNING: there is no transaction in progress
2026-08-09 05:37:21,722 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 05:37:21,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 05:37:21,991 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 05:37:21,992 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 05:37:21,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 05:37:22,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 05:37:22,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-09 05:37:22,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 05:37:23,024 [I] Table x_portal_user does not exist in database ranger
2026-08-09 05:37:23,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 05:37:23,281 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 05:37:23,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 05:37:23,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 05:37:23,785 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 05:37:23,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 05:37:24.752 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.768 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.784 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.800 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.816 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.833 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.886 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.893 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.908 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.919 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.929 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.944 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.955 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.962 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.969 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:24.976 UTC [17086] WARNING: there is no transaction in progress
2026-08-09 05:37:25,606 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 05:37:25,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 05:37:25,868 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 05:37:25,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 05:37:26,145 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 05:37:26,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 05:37:26,401 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 05:37:26,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 05:37:26,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 05:37:26,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-09 05:37:27,160 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 05:37:27,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 05:37:27,428 [I] DB_PATCHES have already been applied
I20260809 05:37:27.434994 31271 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-admin:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskUEwy2v/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 05:37:27.435088 31271 mini_ranger.cc:193] Using host: 127.30.137.212
I20260809 05:37:27.456188 31271 mini_ranger.cc:227] Ranger admin URL: http://127.30.137.212:39037
Aug 09, 2026 5:37:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 5:37:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 5:37:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 5:37:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 5:37:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 5:37:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 5:37:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 05:37:28.873029 31271 mini_ranger.cc:160] Time spent starting Ranger: real 9.071s user 0.008s sys 0.057s
Aug 09, 2026 5:37:50 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 5:37:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 5:37:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 5:37:50 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 5:37:51 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 05:37:51.738484 31271 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.137.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.137.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/rangerkms_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.137.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/rangerkms_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.137.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.137.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.137.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/HTTP_127.30.137.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 05:37:51 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253871, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 05:37:51.945853 31271 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 05:37:51.974781 31271 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 05:37:52.768946 31271 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 05:37:52.769081 31271 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms
I20260809 05:37:52.769137 31271 mini_ranger_kms.cc:227] Using postgres at 127.30.137.212:55101
2026-08-09 05:37:53,147 [I] DB FLAVOR :POSTGRES
2026-08-09 05:37:53,148 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 05:37:53,148 [I] Checking connection
2026-08-09 05:37:53,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 05:37:53,416 [I] connection success
2026-08-09 05:37:53,416 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 05:37:53,416 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 05:37:53,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 05:37:53,660 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 05:37:53,661 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 05:37:53,661 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 05:37:53,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.137.212:55101/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 05:37:53,923 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 05:37:54.424681 31271 mini_ranger_kms.cc:310] Created kms service
I20260809 05:37:54.629235 31271 mini_ranger_kms.cc:326] Created kudu user
I20260809 05:37:54.693461 31271 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 05:37:54 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253871, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:55.153759 31271 mini_ranger_kms.cc:384] Added ranger policy
I20260809 05:37:55.154134 31271 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms:/tmp/dist-test-taskUEwy2v/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskUEwy2v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 05:37:55.154198 31271 mini_ranger_kms.cc:240] Using host: 127.30.137.212
I20260809 05:37:55.176122 31271 mini_ranger_kms.cc:276] Ranger KMS PID: 17359
I20260809 05:37:55.176275 31271 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.137.212:38481
Aug 09, 2026 5:37:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 5:37:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 5:37:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 5:37:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 5:37:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 5:37:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 05:37:57 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786253877, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.137.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 05:37:57 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253877, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.137.212@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:37:58.174806 31271 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.229s user 0.069s sys 0.351s
I20260809 05:37:58.175103 31271 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 05:37:58.175344 31271 mini_ranger_kms.cc:401] 127.30.137.212:38481/kms/v1/keys
Aug 09 05:37:58 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253877, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.137.212@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
Aug 09 05:38:00 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:38:00.863154 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43741
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43741
--ranger_config_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:01.231964 17427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:01.232401 17427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:01.232576 17427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:01.240397 17427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:01.240586 17427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:01.240708 17427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:01.240809 17427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:01.240900 17427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:01.251979 17427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.137.212:38481/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43741
--ranger_config_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:01.253427 17427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:01.254973 17427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:01.269294 17441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:01.269481 17435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:01.269294 17436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:01.272336 17427 server_base.cc:1034] running on GCE node
I20260809 05:38:01.273416 17427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:01.275251 17427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:01.276543 17427 hybrid_clock.cc:648] HybridClock initialized: now 1786253881276479 us; error 64 us; skew 500 ppm
I20260809 05:38:01.277007 17427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:01 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253881, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:01.285552 17427 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:01.287480 17427 webserver.cc:460] Webserver started at http://127.30.137.254:36713/ using document root <none> and password file <none>
I20260809 05:38:01.288249 17427 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:01.288411 17427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:01.288748 17427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:38:01 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253881, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:01.365664 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "7dbdded4396b4fb2a3e7486aa746476c"
format_stamp: "Formatted at 2026-08-09 05:38:01 on dist-test-slave-3kk6"
server_key: "6a98dabab9d385365c7185fea88b482d"
server_key_iv: "01219c55b76454d4c2cf2cd3a97c9dca"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:01.366741 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "7dbdded4396b4fb2a3e7486aa746476c"
format_stamp: "Formatted at 2026-08-09 05:38:01 on dist-test-slave-3kk6"
server_key: "6a98dabab9d385365c7185fea88b482d"
server_key_iv: "01219c55b76454d4c2cf2cd3a97c9dca"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:01.373242 17427 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 05:38:01.394259 17450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:01.396083 17427 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:38:01.396319 17427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "7dbdded4396b4fb2a3e7486aa746476c"
format_stamp: "Formatted at 2026-08-09 05:38:01 on dist-test-slave-3kk6"
server_key: "6a98dabab9d385365c7185fea88b482d"
server_key_iv: "01219c55b76454d4c2cf2cd3a97c9dca"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:01.396582 17427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:01.416776 17427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:01.421998 17427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:01.422484 17427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:01.462623 17427 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:43741
I20260809 05:38:01.462630 17524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:43741 every 8 connection(s)
I20260809 05:38:01.464957 17427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:01.469709 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 17427
I20260809 05:38:01.469946 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:38:01.470855 31271 external_mini_cluster.cc:1428] Setting key 40b2f09093f9af1c765bafd482a16207
I20260809 05:38:01.471236 17525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:38:01 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:01.494326 17525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Bootstrap starting.
I20260809 05:38:01.499702 17525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:01.501611 17525 log.cc:824] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:01.504577 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:01.473836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.30.137.254:43741)
0809 05:38:01.474808 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:01.474824 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:01.474878 (+ 54us) server_negotiation.cc:407] Connection header received
0809 05:38:01.477858 (+ 2980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:01.477894 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:01.478483 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:01.480918 (+ 2435us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:01.481528 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:01.483924 (+ 2396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:01.486015 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:01.487189 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:01.488221 (+ 1032us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:01.493122 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:01.493134 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:01.493157 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:01.493210 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:01.497553 (+ 4343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:01.498642 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:01.498675 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:01.498685 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:01.498903 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:01.499518 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:01.499523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:01.499527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:01.500446 (+ 919us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:01.502170 (+ 1724us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:01.502508 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 05:38:01.503043 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":482,"threads_started":1}
I20260809 05:38:01.506140 17525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: No bootstrap required, opened a new log
I20260809 05:38:01.514514 17525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:01.515061 17525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:01.515199 17525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbdded4396b4fb2a3e7486aa746476c, State: Initialized, Role: FOLLOWER
I20260809 05:38:01.515872 17525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:01.516131 17525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:01.516302 17525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:01.516503 17525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:01.519357 17525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:01.519941 17525 leader_election.cc:302] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dbdded4396b4fb2a3e7486aa746476c; no voters:
I20260809 05:38:01.520700 17525 leader_election.cc:288] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:38:01.520838 17530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:01.523818 17525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:01.527801 17530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 LEADER]: Becoming Leader. State: Replica: 7dbdded4396b4fb2a3e7486aa746476c, State: Running, Role: LEADER
I20260809 05:38:01.528615 17530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:01.532415 17532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dbdded4396b4fb2a3e7486aa746476c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } } }
I20260809 05:38:01.532749 17532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:01.533304 17535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dbdded4396b4fb2a3e7486aa746476c. Latest consensus state: current_term: 1 leader_uuid: "7dbdded4396b4fb2a3e7486aa746476c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } } }
I20260809 05:38:01.533823 17535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:01.547685 17525 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 05:38:01.549889 17541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:01.564904 17541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:01.581591 17541 catalog_manager.cc:1347] Generated new cluster ID: e9d37374607041ffa38f2d67ca2d9eeb
I20260809 05:38:01.581740 17541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:01.592247 17541 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:38:01.593755 17541 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:01.599479 17541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Generated new TSK 0
I20260809 05:38:01.600674 17541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:38:02.399201 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43741
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:38:02 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786253882, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 05:38:02.895862 17583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:02.896364 17583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:02.896595 17583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:02.908924 17583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:02.909102 17583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:02.909368 17583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:38:02.922340 17583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.137.212:38481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:02.923902 17583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:02.925542 17583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:02.946938 17599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:02.951052 17600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:02.952872 17602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:02.956723 17583 server_base.cc:1034] running on GCE node
I20260809 05:38:02.957609 17583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:02.963846 17583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:02.965252 17583 hybrid_clock.cc:648] HybridClock initialized: now 1786253882965135 us; error 122 us; skew 500 ppm
I20260809 05:38:02.965775 17583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:02 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253882, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:02.970681 17583 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:38:02.973052 17583 webserver.cc:460] Webserver started at http://127.30.137.193:44931/ using document root <none> and password file <none>
I20260809 05:38:02.974087 17583 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:02.974326 17583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:02.974803 17583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:38:02 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253882, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:03.003207 17583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "3064b64d810043b7b42843c72c236e1a"
format_stamp: "Formatted at 2026-08-09 05:38:02 on dist-test-slave-3kk6"
server_key: "cc4d9f22559c858639ee8dd29bd8e7c5"
server_key_iv: "89ba8ac13d34f02561b52d716ca3ea17"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:03.004613 17583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "3064b64d810043b7b42843c72c236e1a"
format_stamp: "Formatted at 2026-08-09 05:38:02 on dist-test-slave-3kk6"
server_key: "cc4d9f22559c858639ee8dd29bd8e7c5"
server_key_iv: "89ba8ac13d34f02561b52d716ca3ea17"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:03.023447 17583 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.013s sys 0.004s
I20260809 05:38:03.063397 17615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:03.065572 17583 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.003s
I20260809 05:38:03.065894 17583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "3064b64d810043b7b42843c72c236e1a"
format_stamp: "Formatted at 2026-08-09 05:38:02 on dist-test-slave-3kk6"
server_key: "cc4d9f22559c858639ee8dd29bd8e7c5"
server_key_iv: "89ba8ac13d34f02561b52d716ca3ea17"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:03.066340 17583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:03.110836 17583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:03.116735 17583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:03.117167 17583 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 09 05:38:03 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253882, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:03.124655 17583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:03.130117 17583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:03.130295 17583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 05:38:03.130493 17583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:03.130607 17583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:03.257751 17583 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:34983
I20260809 05:38:03.257791 17779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:34983 every 8 connection(s)
I20260809 05:38:03.259857 17583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:38:03.264308 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 17583
I20260809 05:38:03.264590 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:38:03.265750 31271 external_mini_cluster.cc:1428] Setting key e667b5087fb6afac13c4a7f8b1f2cdef
Aug 09 05:38:03 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253882, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:03.303580 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:03.263165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:33155 (local address 127.30.137.254:43741)
0809 05:38:03.278361 (+ 15196us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:03.278368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:03.282593 (+ 4225us) server_negotiation.cc:407] Connection header received
0809 05:38:03.282711 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:03.282716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:03.282846 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:03.282971 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:03.285331 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:03.286509 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:03.289146 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:03.289868 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:03.289975 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:03.295043 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:03.295052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:03.295057 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:03.295112 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:03.299119 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:03.300230 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:03.300236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:03.300240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:03.300405 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:03.301092 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:03.301097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:03.301100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:03.301490 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:03.301640 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:03.302905 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0809 05:38:03.303176 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15039,"thread_start_us":19258,"threads_started":1}
I20260809 05:38:03.306479 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:03.263552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43741 (local address 127.30.137.193:33155)
0809 05:38:03.279119 (+ 15567us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:03.279196 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:03.281995 (+ 2799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:03.283481 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:03.283495 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:03.284193 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:03.285091 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:03.285119 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:03.286821 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:03.286827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:03.288960 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:03.288976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:03.290360 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:03.290369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:03.290729 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:03.291696 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:03.291729 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:03.294717 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:03.299362 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:03.299376 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:03.299400 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:03.300063 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:03.300565 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:03.300570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:03.300574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:03.300935 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:03.301815 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:03.301823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:03.302365 (+ 542us) client_negotiation.cc:769] Sending connection context
0809 05:38:03.304201 (+ 1836us) client_negotiation.cc:240] Negotiation successful
0809 05:38:03.304846 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15164,"thread_start_us":15752,"threads_started":1}
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
I20260809 05:38:03.308957 17780 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43741
I20260809 05:38:03.309384 17780 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:03.310382 17780 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
I20260809 05:38:03.313494 17473 ts_manager.cc:194] Registered new tserver with Master: 3064b64d810043b7b42843c72c236e1a (127.30.137.193:34983)
I20260809 05:38:03.315685 17473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:33155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:38:03.354625 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43741
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:38:03 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253882, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
W20260809 05:38:03.843538 17790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:03.844031 17790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:03.844190 17790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:03.851866 17790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:03.851985 17790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:03.852161 17790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:38:03.862269 17790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.137.212:38481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:03.863755 17790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:03.865116 17790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:03.879586 17811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:03.879689 17815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:03.880260 17813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:03.881150 17790 server_base.cc:1034] running on GCE node
I20260809 05:38:03.881904 17790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:03.883164 17790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:03.884420 17790 hybrid_clock.cc:648] HybridClock initialized: now 1786253883884364 us; error 49 us; skew 500 ppm
I20260809 05:38:03.884802 17790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:03 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253883, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:03.889041 17790 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:38:03.891008 17790 webserver.cc:460] Webserver started at http://127.30.137.194:32979/ using document root <none> and password file <none>
I20260809 05:38:03.891983 17790 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:03.892128 17790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:03.892494 17790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:38:03 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253883, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:03.909327 17790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "39a18dccdd1e4a13b80a327431a8273c"
format_stamp: "Formatted at 2026-08-09 05:38:03 on dist-test-slave-3kk6"
server_key: "5d3955dd5a87a093a5ae52e4cbcb4c10"
server_key_iv: "d823f5cefec83a9efb6da2a13a068dd4"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:03.910282 17790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "39a18dccdd1e4a13b80a327431a8273c"
format_stamp: "Formatted at 2026-08-09 05:38:03 on dist-test-slave-3kk6"
server_key: "5d3955dd5a87a093a5ae52e4cbcb4c10"
server_key_iv: "d823f5cefec83a9efb6da2a13a068dd4"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:03.916397 17790 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260809 05:38:03.940579 17824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:03.942804 17790 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.008s
I20260809 05:38:03.943035 17790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "39a18dccdd1e4a13b80a327431a8273c"
format_stamp: "Formatted at 2026-08-09 05:38:03 on dist-test-slave-3kk6"
server_key: "5d3955dd5a87a093a5ae52e4cbcb4c10"
server_key_iv: "d823f5cefec83a9efb6da2a13a068dd4"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:03.943310 17790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:03.992565 17790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:03.997834 17790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:03.998193 17790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:03.999421 17790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:04.001303 17790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:04.001401 17790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:04.001528 17790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:04.001590 17790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:04.040289 17790 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:41155
I20260809 05:38:04.040308 17989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:41155 every 8 connection(s)
I20260809 05:38:04.041990 17790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:38:04.048873 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 17790
I20260809 05:38:04.049121 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:38:04.049787 31271 external_mini_cluster.cc:1428] Setting key 77137ff770ad8ab98f8478cee1e1663a
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253883, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:04.071938 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.046736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:50611 (local address 127.30.137.254:43741)
0809 05:38:04.047221 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:04.047226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:04.048372 (+ 1146us) server_negotiation.cc:407] Connection header received
0809 05:38:04.051301 (+ 2929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:04.051308 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:04.051442 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:04.051597 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:04.053676 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.054894 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.058101 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.058824 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.058982 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.063817 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:04.063827 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:04.063834 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:04.063888 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:04.067395 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.068383 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.068392 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.068399 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.068567 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.069434 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.069442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.069448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.069881 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:04.070055 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:04.071015 (+ 960us) server_negotiation.cc:299] Negotiation successful
0809 05:38:04.071342 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":263,"threads_started":1}
I20260809 05:38:04.073010 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.047403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43741 (local address 127.30.137.194:50611)
0809 05:38:04.048197 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:04.048237 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:04.050868 (+ 2631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:04.051951 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:04.051963 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:04.052538 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:04.053447 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:04.053477 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.055669 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.055677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:04.057910 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:04.057930 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.059117 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.059128 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:04.059490 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.060400 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:04.060437 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:04.063529 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:04.067571 (+ 4042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:04.067581 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:04.067601 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:04.068224 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:04.068990 (+ 766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:04.068995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:04.068999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:04.069291 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:04.070046 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:04.070054 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:04.070516 (+ 462us) client_negotiation.cc:769] Sending connection context
0809 05:38:04.070885 (+ 369us) client_negotiation.cc:240] Negotiation successful
0809 05:38:04.071445 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":340,"threads_started":1}
I20260809 05:38:04.074939 17990 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43741
I20260809 05:38:04.075310 17990 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:04.076277 17990 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
I20260809 05:38:04.078073 17473 ts_manager.cc:194] Registered new tserver with Master: 39a18dccdd1e4a13b80a327431a8273c (127.30.137.194:41155)
I20260809 05:38:04.079186 17473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:50611
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:38:04.127341 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43741
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:04.319942 17780 heartbeater.cc:503] Master 127.30.137.254:43741 was elected leader, sending a full tablet report...
W20260809 05:38:04.491189 17997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:04.491559 17997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:04.491760 17997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:04.499405 17997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:04.499586 17997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:04.499792 17997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:38:04.510360 17997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.137.212:38481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:04.511880 17997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:04.513267 17997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:04.527472 18009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:04.527535 18006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:04.527477 18013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:04.528479 17997 server_base.cc:1034] running on GCE node
I20260809 05:38:04.529191 17997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:04.530321 17997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:04.531594 17997 hybrid_clock.cc:648] HybridClock initialized: now 1786253884531548 us; error 41 us; skew 500 ppm
I20260809 05:38:04.531989 17997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253884, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:04.535544 17997 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:38:04.537340 17997 webserver.cc:460] Webserver started at http://127.30.137.195:34777/ using document root <none> and password file <none>
I20260809 05:38:04.538151 17997 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:04.538270 17997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:04.538540 17997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253884, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:04.555326 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "b8224e57775249dab7765e4a648482b3"
format_stamp: "Formatted at 2026-08-09 05:38:04 on dist-test-slave-3kk6"
server_key: "54a8e31a6854fa1cc9311175d46ddbcd"
server_key_iv: "075c07ce1eb973347fafc100e3759424"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:04.556353 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "b8224e57775249dab7765e4a648482b3"
format_stamp: "Formatted at 2026-08-09 05:38:04 on dist-test-slave-3kk6"
server_key: "54a8e31a6854fa1cc9311175d46ddbcd"
server_key_iv: "075c07ce1eb973347fafc100e3759424"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:04.562496 17997 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 05:38:04.578127 18021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:04.579918 17997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:04.580125 17997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "b8224e57775249dab7765e4a648482b3"
format_stamp: "Formatted at 2026-08-09 05:38:04 on dist-test-slave-3kk6"
server_key: "54a8e31a6854fa1cc9311175d46ddbcd"
server_key_iv: "075c07ce1eb973347fafc100e3759424"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:04.580367 17997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:04.606290 17997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:04.610809 17997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:04.611151 17997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:04.612264 17997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:04.613795 17997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:04.613873 17997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:04.614007 17997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:04.614061 17997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:04.650610 17997 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40449
I20260809 05:38:04.650733 18183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40449 every 8 connection(s)
I20260809 05:38:04.652145 17997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:38:04.661268 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 17997
I20260809 05:38:04.661480 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:38:04.662031 31271 external_mini_cluster.cc:1428] Setting key 7e82c930427ed036e31b3b5ffe47f1e7
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253884, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:04.674716 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.655193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:43525 (local address 127.30.137.254:43741)
0809 05:38:04.655685 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:04.655692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:04.656811 (+ 1119us) server_negotiation.cc:407] Connection header received
0809 05:38:04.658936 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:04.658942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:04.659063 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:04.659201 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:04.660760 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.661607 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.663473 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.664198 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.664340 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.668198 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:04.668203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:04.668207 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:04.668241 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:04.671256 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.672063 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.672068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.672090 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.672280 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.672853 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.672857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.672859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.673240 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:04.673409 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:04.674183 (+ 774us) server_negotiation.cc:299] Negotiation successful
0809 05:38:04.674369 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":243,"threads_started":1}
I20260809 05:38:04.675721 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.655825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43741 (local address 127.30.137.195:43525)
0809 05:38:04.656587 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:04.656626 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:04.658576 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:04.659446 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:04.659455 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:04.659921 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:04.660525 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:04.660548 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.661772 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.661776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:04.663291 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:04.663304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.664493 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.664504 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:04.664821 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.665737 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:04.665761 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:04.667987 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:04.671418 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:04.671425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:04.671439 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:04.671925 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:04.672395 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:04.672398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:04.672400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:04.672723 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:04.673415 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:04.673425 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:04.673804 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 05:38:04.674047 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 05:38:04.674454 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":316,"threads_started":1}
I20260809 05:38:04.677384 18186 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43741
I20260809 05:38:04.677754 18186 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:04.678622 18186 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
I20260809 05:38:04.680302 17471 ts_manager.cc:194] Registered new tserver with Master: b8224e57775249dab7765e4a648482b3 (127.30.137.195:40449)
I20260809 05:38:04.681211 17471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:43525
I20260809 05:38:04.691160 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:38:04.692947 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 17583
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:38:04.711467 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.693923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.30.137.193:34983)
0809 05:38:04.694820 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:04.694829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:04.694858 (+ 29us) server_negotiation.cc:407] Connection header received
0809 05:38:04.694919 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:04.694925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:04.695146 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:04.695306 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:04.696885 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.698038 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.699576 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.700260 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.700379 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.705218 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:04.705232 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:04.705246 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:04.705290 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:04.708366 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.709146 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.709155 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.709157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.709319 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.709715 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.709721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.709726 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.710088 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:04.710439 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:04.710777 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 05:38:04.711074 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":482,"threads_started":1}
I20260809 05:38:04.712838 17774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:04.799187 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 17583
I20260809 05:38:04.824445 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 17790
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:38:04.842007 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.825488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52170 (local address 127.30.137.194:41155)
0809 05:38:04.826188 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:04.826196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:04.826227 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:38:04.826294 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:04.826301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:04.826519 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:04.826756 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:04.828023 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.829135 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.830422 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.831218 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.831325 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.836293 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:04.836305 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:04.836320 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:04.836360 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:04.839475 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.840162 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.840169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.840172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.840311 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.840635 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.840639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.840640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.840991 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:04.841235 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:04.841494 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 05:38:04.841698 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":410,"threads_started":1}
I20260809 05:38:04.843642 17983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:04.929540 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 17790
I20260809 05:38:04.961287 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 17997
Aug 09 05:38:04 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:38:04.980919 18201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:04.962434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.30.137.195:40449)
0809 05:38:04.963321 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:04.963329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:04.963381 (+ 52us) server_negotiation.cc:407] Connection header received
0809 05:38:04.963498 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:04.963512 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:04.963819 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:04.964022 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:04.966065 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.967345 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.968814 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:04.969433 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:04.969545 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:04.974415 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:04.974431 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:04.974448 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:04.974490 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:04.977632 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.978448 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.978456 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.978459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.978614 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:04.979281 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:04.979306 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:04.979308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:04.979727 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:04.979987 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:04.980340 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 05:38:04.980569 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":455,"threads_started":1}
I20260809 05:38:04.982367 18178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:05.066720 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 17997
I20260809 05:38:05.092195 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 17427
I20260809 05:38:05.093549 17517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:05.202802 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 17427
I20260809 05:38:05.237608 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43741
--webserver_interface=127.30.137.254
--webserver_port=36713
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43741
--ranger_config_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:05.586575 18204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:05.587001 18204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:05.587177 18204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:05.594818 18204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:05.594976 18204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:05.595079 18204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:05.595185 18204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:05.595281 18204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:05.605798 18204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43741
--ranger_config_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=36713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:05.607331 18204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:05.608752 18204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:05.621407 18210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:05.621459 18213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:05.621482 18211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:05.622644 18204 server_base.cc:1034] running on GCE node
I20260809 05:38:05.623538 18204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:05.624967 18204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:05.626215 18204 hybrid_clock.cc:648] HybridClock initialized: now 1786253885626177 us; error 58 us; skew 500 ppm
I20260809 05:38:05.626607 18204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:05 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253885, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:05.630091 18204 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:05.631772 18204 webserver.cc:460] Webserver started at http://127.30.137.254:36713/ using document root <none> and password file <none>
I20260809 05:38:05.632459 18204 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:05.632615 18204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 05:38:05 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253885, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:05.683924 18204 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 05:38:05.687116 18225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:05.688591 18204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 05:38:05.688828 18204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "7dbdded4396b4fb2a3e7486aa746476c"
format_stamp: "Formatted at 2026-08-09 05:38:01 on dist-test-slave-3kk6"
server_key: "6a98dabab9d385365c7185fea88b482d"
server_key_iv: "01219c55b76454d4c2cf2cd3a97c9dca"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:05.689883 18204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:05.717569 18204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:05.721748 18204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:05.722072 18204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:05.742524 18204 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:43741
I20260809 05:38:05.742547 18302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:43741 every 8 connection(s)
I20260809 05:38:05.744261 18204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:05.748529 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 18204
I20260809 05:38:05.749691 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:34983
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:43741
--webserver_port=44931
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:05.753111 18306 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:38:05.758734 18306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Bootstrap starting.
I20260809 05:38:05.779179 18306 log.cc:824] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:05.788654 18306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:38:05.789255 18306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Bootstrap complete.
I20260809 05:38:05.799208 18306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:05.800076 18306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbdded4396b4fb2a3e7486aa746476c, State: Initialized, Role: FOLLOWER
I20260809 05:38:05.800765 18306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:05.801007 18306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:05.801129 18306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:05.801318 18306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:38:05.805716 18306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:05.806253 18306 leader_election.cc:302] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dbdded4396b4fb2a3e7486aa746476c; no voters:
I20260809 05:38:05.806993 18306 leader_election.cc:288] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 05:38:05.807142 18310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 05:38:05.808441 18310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [term 2 LEADER]: Becoming Leader. State: Replica: 7dbdded4396b4fb2a3e7486aa746476c, State: Running, Role: LEADER
I20260809 05:38:05.809094 18310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } }
I20260809 05:38:05.809401 18306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:05.812084 18311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7dbdded4396b4fb2a3e7486aa746476c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } } }
I20260809 05:38:05.812609 18311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:05.812073 18312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dbdded4396b4fb2a3e7486aa746476c. Latest consensus state: current_term: 2 leader_uuid: "7dbdded4396b4fb2a3e7486aa746476c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbdded4396b4fb2a3e7486aa746476c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43741 } } }
I20260809 05:38:05.813149 18312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:05.814172 18319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:05.828326 18319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:05.830343 18319 catalog_manager.cc:1259] Loaded cluster ID: e9d37374607041ffa38f2d67ca2d9eeb
I20260809 05:38:05.830591 18319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:05.836045 18319 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:05.838081 18319 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Loaded TSK: 0
I20260809 05:38:05.840212 18319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:38:05.843762 18306 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 05:38:06.242971 18308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:06.243487 18308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:06.243709 18308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:06.255831 18308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:06.256008 18308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:06.256283 18308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:38:06.272737 18308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:34983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=44931
--tserver_master_addrs=127.30.137.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:06.274799 18308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:06.276741 18308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:06.320178 18356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:06.323159 18359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:06.327257 18363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:06.329466 18308 server_base.cc:1034] running on GCE node
I20260809 05:38:06.330572 18308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:06.341377 18308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:06.344918 18308 hybrid_clock.cc:648] HybridClock initialized: now 1786253886344815 us; error 102 us; skew 500 ppm
I20260809 05:38:06.345504 18308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:06 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253886, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:06.355453 18308 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:38:06.385226 18308 webserver.cc:460] Webserver started at http://127.30.137.193:44931/ using document root <none> and password file <none>
I20260809 05:38:06.386533 18308 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:06.386853 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 05:38:06 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253886, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:06.442524 18308 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.005s sys 0.001s
I20260809 05:38:06.448608 18375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:06.450520 18308 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.005s
I20260809 05:38:06.450755 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "3064b64d810043b7b42843c72c236e1a"
format_stamp: "Formatted at 2026-08-09 05:38:02 on dist-test-slave-3kk6"
server_key: "cc4d9f22559c858639ee8dd29bd8e7c5"
server_key_iv: "89ba8ac13d34f02561b52d716ca3ea17"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:06.452250 18308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:06.485674 18308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:06.491449 18308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:06.491856 18308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:06.493839 18308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:06.497047 18308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:06.497174 18308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:06.497291 18308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:06.497377 18308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:06.556599 18308 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:34983
I20260809 05:38:06.556638 18546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:34983 every 8 connection(s)
I20260809 05:38:06.558617 18308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:38:06.564033 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 18308
I20260809 05:38:06.565006 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:41155
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:43741
--webserver_port=32979
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:38:06 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253886, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:06.592622 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:06.562110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:51257 (local address 127.30.137.254:43741)
0809 05:38:06.563859 (+ 1749us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:06.563880 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:06.563943 (+ 63us) server_negotiation.cc:407] Connection header received
0809 05:38:06.567536 (+ 3593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:06.567560 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:06.568140 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:06.568583 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:06.571281 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:06.573384 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:06.576052 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:06.577182 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:06.577365 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:06.582223 (+ 4858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:06.582234 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:06.582255 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:06.582305 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:06.586476 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:06.587568 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:06.587578 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:06.587590 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:06.587819 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:06.588431 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:06.588436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:06.588439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:06.589137 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:06.589342 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:06.590568 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0809 05:38:06.591068 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1482,"thread_start_us":1229,"threads_started":1}
I20260809 05:38:06.595455 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:06.562055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43741 (local address 127.30.137.193:51257)
0809 05:38:06.562959 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:06.563018 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:06.566534 (+ 3516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:06.569327 (+ 2793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:06.569341 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:06.570073 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:06.570979 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:06.571013 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:06.573606 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:06.573614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:06.575859 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:06.575880 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:06.577677 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:06.577687 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:06.578029 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:06.579008 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:06.579046 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:06.581886 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:06.586709 (+ 4823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:06.586718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:06.586735 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:06.587375 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:06.588012 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:06.588017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:06.588020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:06.588288 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:06.589446 (+ 1158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:06.589455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:06.589938 (+ 483us) client_negotiation.cc:769] Sending connection context
0809 05:38:06.593331 (+ 3393us) client_negotiation.cc:240] Negotiation successful
0809 05:38:06.593916 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":377,"threads_started":1}
I20260809 05:38:06.599813 18548 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43741
I20260809 05:38:06.600160 18548 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:06.601125 18548 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
W20260809 05:38:06.604874 18548 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:43741 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 05:38:06.605125 18548 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:06.605418 18548 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
I20260809 05:38:06.606892 18548 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:06.607271 18548 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
W20260809 05:38:06.608440 18548 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 05:38:07.049166 18552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:07.049733 18552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:07.050009 18552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:07.061932 18552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:07.062165 18552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:07.062450 18552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:38:07.078629 18552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:41155
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=32979
--tserver_master_addrs=127.30.137.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:07.080401 18552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:07.082135 18552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:07.099838 18578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:07.101763 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:07.106217 18584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:07.106516 18552 server_base.cc:1034] running on GCE node
I20260809 05:38:07.107600 18552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:07.109284 18552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:07.110667 18552 hybrid_clock.cc:648] HybridClock initialized: now 1786253887110574 us; error 64 us; skew 500 ppm
I20260809 05:38:07.111066 18552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:07.115697 18552 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:38:07.117844 18552 webserver.cc:460] Webserver started at http://127.30.137.194:32979/ using document root <none> and password file <none>
I20260809 05:38:07.118916 18552 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:07.119102 18552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:07.165011 18552 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:38:07.171932 18594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:07.173763 18552 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.000s
I20260809 05:38:07.173969 18552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "39a18dccdd1e4a13b80a327431a8273c"
format_stamp: "Formatted at 2026-08-09 05:38:03 on dist-test-slave-3kk6"
server_key: "5d3955dd5a87a093a5ae52e4cbcb4c10"
server_key_iv: "d823f5cefec83a9efb6da2a13a068dd4"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:07.175439 18552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:07.195413 18552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:07.201087 18552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:07.201445 18552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:07.202839 18552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:07.204759 18552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:07.204890 18552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:07.205008 18552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:07.205088 18552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:07.265281 18552 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:41155
I20260809 05:38:07.265342 18758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:41155 every 8 connection(s)
I20260809 05:38:07.266960 18552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:38:07.271037 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 18552
I20260809 05:38:07.272112 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.137.212:38481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:40449
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:43741
--webserver_port=34777
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:07.313730 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:07.270293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:58343 (local address 127.30.137.254:43741)
0809 05:38:07.289040 (+ 18747us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:07.289051 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:07.289723 (+ 672us) server_negotiation.cc:407] Connection header received
0809 05:38:07.292870 (+ 3147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:07.292878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:07.292979 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:07.293097 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:07.295724 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:07.296935 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:07.299457 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:07.300136 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:07.300276 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:07.305383 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:07.305390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:07.305395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:07.305437 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:07.309453 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:07.310509 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:07.310515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:07.310518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:07.310848 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:07.311481 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:07.311488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:07.311491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:07.311897 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:07.312044 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:07.313084 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0809 05:38:07.313343 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18574,"thread_start_us":18453,"threads_started":1}
I20260809 05:38:07.316514 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:07.270959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43741 (local address 127.30.137.194:58343)
0809 05:38:07.289452 (+ 18493us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:07.289545 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:07.292338 (+ 2793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:07.293632 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:07.293645 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:07.294412 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:07.295481 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:07.295513 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:07.297159 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:07.297164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:07.299269 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:07.299286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:07.300706 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:07.300715 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:07.301036 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:07.301942 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:07.301977 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:07.305087 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:07.309642 (+ 4555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:07.309652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:07.309671 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:07.310350 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:07.311020 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:07.311025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:07.311029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:07.311330 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:07.312170 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:07.312177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:07.312593 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 05:38:07.314358 (+ 1765us) client_negotiation.cc:240] Negotiation successful
0809 05:38:07.315005 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18217,"thread_start_us":17989,"threads_started":1}
I20260809 05:38:07.318740 18760 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43741
I20260809 05:38:07.319088 18760 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:07.320093 18760 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
I20260809 05:38:07.322435 18247 ts_manager.cc:194] Registered new tserver with Master: 39a18dccdd1e4a13b80a327431a8273c (127.30.137.194:41155)
I20260809 05:38:07.324776 18247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:58343
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:07.608621 18548 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:07.609141 18548 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
I20260809 05:38:07.610641 18247 ts_manager.cc:194] Registered new tserver with Master: 3064b64d810043b7b42843c72c236e1a (127.30.137.193:34983)
I20260809 05:38:07.611950 18247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:51257
W20260809 05:38:07.791185 18766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:07.791639 18766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:07.791848 18766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:07.804901 18766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:07.805073 18766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:07.805370 18766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:38:07.823210 18766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.137.212:38481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:40449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=34777
--tserver_master_addrs=127.30.137.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:07.825081 18766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:07.826992 18766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:07.849732 18776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:07.851186 18777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:07.858249 18780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:07.859133 18766 server_base.cc:1034] running on GCE node
I20260809 05:38:07.860055 18766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:07.861600 18766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:07.862923 18766 hybrid_clock.cc:648] HybridClock initialized: now 1786253887862814 us; error 108 us; skew 500 ppm
I20260809 05:38:07.863358 18766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:07.867728 18766 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:38:07.869812 18766 webserver.cc:460] Webserver started at http://127.30.137.195:34777/ using document root <none> and password file <none>
I20260809 05:38:07.870859 18766 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:07.871030 18766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:07.909242 18766 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.008s sys 0.004s
I20260809 05:38:07.922614 18792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:07.925019 18766 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.011s
I20260809 05:38:07.925304 18766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "b8224e57775249dab7765e4a648482b3"
format_stamp: "Formatted at 2026-08-09 05:38:04 on dist-test-slave-3kk6"
server_key: "54a8e31a6854fa1cc9311175d46ddbcd"
server_key_iv: "075c07ce1eb973347fafc100e3759424"
server_key_version: "kuduclusterkey@0"
I20260809 05:38:07.926618 18766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 05:38:07 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for HTTP/127.30.137.212@KRBTEST.COM
I20260809 05:38:07.983937 18766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:07.989151 18766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:07.989579 18766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:07.997522 18766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:08.007387 18766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:08.007622 18766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 05:38:08.007802 18766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:08.007956 18766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:08.160090 18766 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40449
I20260809 05:38:08.162156 18766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:38:08.164619 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 18766
I20260809 05:38:08.171799 18947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40449 every 8 connection(s)
I20260809 05:38:08.197366 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.165898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.30.137.254:43741)
0809 05:38:08.169750 (+ 3852us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.169756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.176113 (+ 6357us) server_negotiation.cc:407] Connection header received
0809 05:38:08.176663 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.176670 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.176774 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.176862 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:08.179551 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.180822 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.183160 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.183856 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.183964 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.188310 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:08.188318 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:08.188324 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:08.188410 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:08.192532 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.194146 (+ 1614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.194152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.194155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.194331 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.194866 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.194871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.194875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.195258 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:08.195566 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.196211 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.196478 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3718,"thread_start_us":3595,"threads_started":1}
Aug 09 05:38:08 dist-test-slave-3kk6 krb5kdc[16773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253887, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:08.226626 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.175678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:33787 (local address 127.30.137.254:43741)
0809 05:38:08.197259 (+ 21581us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.197265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.198504 (+ 1239us) server_negotiation.cc:407] Connection header received
0809 05:38:08.202085 (+ 3581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.202091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.202192 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.202315 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:08.204557 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.205858 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.208563 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.209412 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.209540 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.217791 (+ 8251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:08.217802 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:08.217809 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:08.217870 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:08.221904 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.222999 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.223005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.223009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.223171 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.223764 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.223770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.223773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.224171 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:08.224318 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.225973 (+ 1655us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.226196 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21452,"thread_start_us":21605,"threads_started":1}
I20260809 05:38:08.227460 18953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.175121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43741 (local address 127.30.137.195:33787)
0809 05:38:08.198023 (+ 22902us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:08.198093 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:08.200891 (+ 2798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:08.202638 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:08.202681 (+ 43us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:08.203344 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:08.204237 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.204266 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.206073 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.206080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.208308 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.208330 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.209897 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.209906 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.210229 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.211289 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:08.211322 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:08.217405 (+ 6083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:08.222082 (+ 4677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:08.222093 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:08.222112 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:08.222836 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:08.223302 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:08.223306 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:08.223309 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:08.223606 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:08.224411 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:08.224422 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:08.224930 (+ 508us) client_negotiation.cc:769] Sending connection context
0809 05:38:08.225280 (+ 350us) client_negotiation.cc:240] Negotiation successful
0809 05:38:08.225920 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22517,"thread_start_us":25999,"threads_started":1}
I20260809 05:38:08.229297 18948 heartbeater.cc:347] Connected to a master server at 127.30.137.254:43741
I20260809 05:38:08.229662 18948 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:08.230621 18948 heartbeater.cc:511] Master 127.30.137.254:43741 requested a full tablet report, sending...
I20260809 05:38:08.232303 18247 ts_manager.cc:194] Registered new tserver with Master: b8224e57775249dab7765e4a648482b3 (127.30.137.195:40449)
I20260809 05:38:08.233246 18247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:33787
I20260809 05:38:08.239301 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:38:08.258111 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.242993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.30.137.254:43741)
0809 05:38:08.243215 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.243224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.243730 (+ 506us) server_negotiation.cc:407] Connection header received
0809 05:38:08.243916 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.243920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.243981 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.244060 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:08.246051 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.246816 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.248769 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.249197 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.249302 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.252608 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:08.252613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:08.252616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:08.252646 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:08.255251 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.256122 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.256126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.256128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.256283 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.256718 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.256729 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.256732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.256985 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:08.257169 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.257595 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.257836 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:38:08.263463 18247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:38:08.273953 18247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:38:08.319470 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.290228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40449 (local address 127.0.0.1:55208)
0809 05:38:08.302401 (+ 12173us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:08.302472 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:08.302731 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:08.304986 (+ 2255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:08.304999 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:08.305042 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:08.305577 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.305594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.312618 (+ 7024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.312627 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.315664 (+ 3037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.315684 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.318180 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.318187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.318430 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.318520 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 05:38:08.318684 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 05:38:08.318964 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11765,"thread_start_us":12499,"threads_started":1}
I20260809 05:38:08.321372 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.291001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.30.137.195:40449)
0809 05:38:08.304140 (+ 13139us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.304151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.304189 (+ 38us) server_negotiation.cc:407] Connection header received
0809 05:38:08.304302 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.304313 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.304662 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.305835 (+ 1173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:08.306196 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.308020 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.315966 (+ 7946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.317905 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.320550 (+ 2645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.320743 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.320830 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.321004 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12947,"thread_start_us":14119,"threads_started":1}
I20260809 05:38:08.326150 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.292512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:41155 (local address 127.0.0.1:48106)
0809 05:38:08.303427 (+ 10915us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:08.303446 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:08.303558 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:08.309253 (+ 5695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:08.309261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:08.309296 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:08.309881 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.309895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.321125 (+ 11230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.321131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.323199 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.323212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.325261 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.325268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.325454 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.325489 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 05:38:08.325557 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 05:38:08.325726 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10495,"mutex_wait_us":1,"thread_start_us":10353,"threads_started":1}
I20260809 05:38:08.325978 18857 tablet_service.cc:1467] Processing CreateTablet for tablet eb7ba3652fa94cf58015861e53d1fe98 (DEFAULT_TABLE table=test-table [id=a5132c8c947b4081951aca6296d09981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:08.327706 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.290384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48106 (local address 127.30.137.194:41155)
0809 05:38:08.308588 (+ 18204us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.308598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.308628 (+ 30us) server_negotiation.cc:407] Connection header received
0809 05:38:08.308728 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.308735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.308984 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.310097 (+ 1113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:08.310366 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.312392 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.323452 (+ 11060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.324997 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.326839 (+ 1842us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.327073 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.327143 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.327312 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18000,"thread_start_us":17858,"threads_started":1}
I20260809 05:38:08.327711 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb7ba3652fa94cf58015861e53d1fe98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:08.329684 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.303674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:34983 (local address 127.0.0.1:51132)
0809 05:38:08.320022 (+ 16348us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:08.320046 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:08.320163 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:08.320787 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:08.320792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:08.320819 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:08.321006 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.321019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.324309 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.324314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.326255 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.326274 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.328894 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.328902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.329089 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.329131 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 05:38:08.329222 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 05:38:08.329353 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16231,"mutex_wait_us":22,"thread_start_us":16248,"threads_started":1}
I20260809 05:38:08.330001 18760 heartbeater.cc:503] Master 127.30.137.254:43741 was elected leader, sending a full tablet report...
I20260809 05:38:08.330813 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.304089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.30.137.193:34983)
0809 05:38:08.319548 (+ 15459us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.319560 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.320285 (+ 725us) server_negotiation.cc:407] Connection header received
0809 05:38:08.320390 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.320396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.320591 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.320815 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:08.321204 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.324125 (+ 2921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.326445 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.328047 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.328620 (+ 573us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.328775 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.330254 (+ 1479us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.330433 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15243,"thread_start_us":15474,"threads_started":1}
I20260809 05:38:08.332103 18664 tablet_service.cc:1467] Processing CreateTablet for tablet eb7ba3652fa94cf58015861e53d1fe98 (DEFAULT_TABLE table=test-table [id=a5132c8c947b4081951aca6296d09981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:08.333642 18664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb7ba3652fa94cf58015861e53d1fe98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:08.340270 18448 tablet_service.cc:1467] Processing CreateTablet for tablet eb7ba3652fa94cf58015861e53d1fe98 (DEFAULT_TABLE table=test-table [id=a5132c8c947b4081951aca6296d09981]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:08.341718 18448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb7ba3652fa94cf58015861e53d1fe98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:08.351931 18988 tablet_bootstrap.cc:492] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Bootstrap starting.
I20260809 05:38:08.355633 18991 tablet_bootstrap.cc:492] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Bootstrap starting.
I20260809 05:38:08.357376 18988 tablet_bootstrap.cc:654] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:08.359000 18988 log.cc:824] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:08.360765 18991 tablet_bootstrap.cc:654] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:08.362396 18991 log.cc:824] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:08.367394 18996 tablet_bootstrap.cc:492] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Bootstrap starting.
I20260809 05:38:08.369673 18988 tablet_bootstrap.cc:492] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: No bootstrap required, opened a new log
I20260809 05:38:08.370152 18988 ts_tablet_manager.cc:1397] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.011s
I20260809 05:38:08.373879 18996 tablet_bootstrap.cc:654] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:08.375252 18996 log.cc:824] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:08.377161 18991 tablet_bootstrap.cc:492] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: No bootstrap required, opened a new log
I20260809 05:38:08.377514 18991 ts_tablet_manager.cc:1397] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Time spent bootstrapping tablet: real 0.022s user 0.015s sys 0.005s
I20260809 05:38:08.383081 18988 raft_consensus.cc:348] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.383610 18988 raft_consensus.cc:374] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:08.383759 18988 raft_consensus.cc:729] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8224e57775249dab7765e4a648482b3, State: Initialized, Role: FOLLOWER
I20260809 05:38:08.384464 18988 consensus_queue.cc:260] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.385190 18996 tablet_bootstrap.cc:492] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: No bootstrap required, opened a new log
I20260809 05:38:08.385512 18996 ts_tablet_manager.cc:1397] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.003s
I20260809 05:38:08.390232 18991 raft_consensus.cc:348] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.390966 18991 raft_consensus.cc:374] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:08.391150 18991 raft_consensus.cc:729] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39a18dccdd1e4a13b80a327431a8273c, State: Initialized, Role: FOLLOWER
I20260809 05:38:08.392148 18991 consensus_queue.cc:260] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.393635 18948 heartbeater.cc:503] Master 127.30.137.254:43741 was elected leader, sending a full tablet report...
I20260809 05:38:08.394418 18988 ts_tablet_manager.cc:1428] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Time spent starting tablet: real 0.024s user 0.014s sys 0.008s
I20260809 05:38:08.397598 18996 raft_consensus.cc:348] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.398106 18996 raft_consensus.cc:374] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:08.399533 18991 ts_tablet_manager.cc:1428] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Time spent starting tablet: real 0.022s user 0.018s sys 0.000s
I20260809 05:38:08.399623 18996 raft_consensus.cc:729] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3064b64d810043b7b42843c72c236e1a, State: Initialized, Role: FOLLOWER
I20260809 05:38:08.400842 18996 consensus_queue.cc:260] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.402343 18548 heartbeater.cc:503] Master 127.30.137.254:43741 was elected leader, sending a full tablet report...
I20260809 05:38:08.404001 18996 ts_tablet_manager.cc:1428] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Time spent starting tablet: real 0.018s user 0.010s sys 0.004s
W20260809 05:38:08.441221 18952 tablet.cc:2369] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:08.527437 18761 tablet.cc:2369] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:08.561801 18549 tablet.cc:2369] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:38:08.623728 19004 raft_consensus.cc:482] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:38:08.624130 19004 raft_consensus.cc:504] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.626118 19004 leader_election.cc:288] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8224e57775249dab7765e4a648482b3 (127.30.137.195:40449), 3064b64d810043b7b42843c72c236e1a (127.30.137.193:34983)
I20260809 05:38:08.633333 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.626719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:59879 (local address 127.30.137.195:40449)
0809 05:38:08.626908 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.626912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.627496 (+ 584us) server_negotiation.cc:407] Connection header received
0809 05:38:08.627790 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.627800 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.627896 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.627962 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:08.628685 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.629434 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.631403 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.632364 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.632553 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.632628 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.632924 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.633052 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:38:08.633589 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.626535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40449 (local address 127.30.137.194:59879)
0809 05:38:08.627361 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:08.627383 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:08.627552 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:08.628183 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:08.628186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:08.628214 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:08.628532 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.628540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.629571 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.629580 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.631150 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.631164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.632573 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.632580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.632749 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.632779 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 05:38:08.632898 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 05:38:08.633097 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":4,"thread_start_us":458,"threads_started":1}
I20260809 05:38:08.633682 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.627871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:38807 (local address 127.30.137.193:34983)
0809 05:38:08.628101 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.628107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.628343 (+ 236us) server_negotiation.cc:407] Connection header received
0809 05:38:08.628567 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.628578 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.628637 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.628735 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:08.629314 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.630036 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.631841 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.632717 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.632898 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.632977 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.633290 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.633421 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 05:38:08.633826 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.627511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:34983 (local address 127.30.137.194:38807)
0809 05:38:08.628219 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:08.628234 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:08.628410 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:08.628749 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:08.628754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:08.628776 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:08.629160 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.629169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.630180 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.630186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.631691 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:08.631704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.632951 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.632958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:08.633108 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.633130 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 05:38:08.633265 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 05:38:08.633401 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":320,"threads_started":1}
I20260809 05:38:08.634840 18880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb7ba3652fa94cf58015861e53d1fe98" candidate_uuid: "39a18dccdd1e4a13b80a327431a8273c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8224e57775249dab7765e4a648482b3" is_pre_election: true
I20260809 05:38:08.634967 18488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb7ba3652fa94cf58015861e53d1fe98" candidate_uuid: "39a18dccdd1e4a13b80a327431a8273c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3064b64d810043b7b42843c72c236e1a" is_pre_election: true
I20260809 05:38:08.635401 18880 raft_consensus.cc:2454] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39a18dccdd1e4a13b80a327431a8273c in term 0.
I20260809 05:38:08.635504 18488 raft_consensus.cc:2454] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39a18dccdd1e4a13b80a327431a8273c in term 0.
I20260809 05:38:08.636085 18602 leader_election.cc:302] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39a18dccdd1e4a13b80a327431a8273c, b8224e57775249dab7765e4a648482b3; no voters:
I20260809 05:38:08.636489 19004 raft_consensus.cc:2781] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:38:08.636638 19004 raft_consensus.cc:482] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:38:08.636772 19004 raft_consensus.cc:3037] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:08.639240 19004 raft_consensus.cc:504] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.639950 19004 leader_election.cc:288] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [CANDIDATE]: Term 1 election: Requested vote from peers b8224e57775249dab7765e4a648482b3 (127.30.137.195:40449), 3064b64d810043b7b42843c72c236e1a (127.30.137.193:34983)
I20260809 05:38:08.640452 18880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb7ba3652fa94cf58015861e53d1fe98" candidate_uuid: "39a18dccdd1e4a13b80a327431a8273c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8224e57775249dab7765e4a648482b3"
I20260809 05:38:08.640694 18880 raft_consensus.cc:3037] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:08.640743 18488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb7ba3652fa94cf58015861e53d1fe98" candidate_uuid: "39a18dccdd1e4a13b80a327431a8273c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3064b64d810043b7b42843c72c236e1a"
I20260809 05:38:08.640937 18488 raft_consensus.cc:3037] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:08.642858 18880 raft_consensus.cc:2454] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39a18dccdd1e4a13b80a327431a8273c in term 1.
I20260809 05:38:08.643252 18602 leader_election.cc:302] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39a18dccdd1e4a13b80a327431a8273c, b8224e57775249dab7765e4a648482b3; no voters:
I20260809 05:38:08.643328 18488 raft_consensus.cc:2454] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39a18dccdd1e4a13b80a327431a8273c in term 1.
I20260809 05:38:08.643563 19004 raft_consensus.cc:2781] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:08.644068 19004 raft_consensus.cc:686] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 1 LEADER]: Becoming Leader. State: Replica: 39a18dccdd1e4a13b80a327431a8273c, State: Running, Role: LEADER
I20260809 05:38:08.644547 19004 consensus_queue.cc:237] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } }
I20260809 05:38:08.649408 18246 catalog_manager.cc:5495] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c reported cstate change: term changed from 0 to 1, leader changed from <none> to 39a18dccdd1e4a13b80a327431a8273c (127.30.137.194). New cstate: current_term: 1 leader_uuid: "39a18dccdd1e4a13b80a327431a8273c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39a18dccdd1e4a13b80a327431a8273c" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 41155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 } health_report { overall_health: UNKNOWN } } }
I20260809 05:38:08.687731 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.682050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.30.137.194:41155)
0809 05:38:08.682263 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.682272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.682332 (+ 60us) server_negotiation.cc:407] Connection header received
0809 05:38:08.682600 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.682604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.682698 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.682872 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:38:08.683290 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.684075 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.685463 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.685869 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.686004 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.686516 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:38:08.687134 (+ 618us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:38:08.687302 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.687361 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.687441 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:38:08.695808 18488 raft_consensus.cc:1261] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 1 FOLLOWER]: Refusing update from remote peer 39a18dccdd1e4a13b80a327431a8273c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:08.695966 18880 raft_consensus.cc:1261] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 1 FOLLOWER]: Refusing update from remote peer 39a18dccdd1e4a13b80a327431a8273c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:08.696908 19013 consensus_queue.cc:1035] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3064b64d810043b7b42843c72c236e1a" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:08.697291 19004 consensus_queue.cc:1035] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8224e57775249dab7765e4a648482b3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:08.710753 19016 mvcc.cc:205] Tried to move back new op lower bound from 7316495928088350720 to 7316495927892135936. Current Snapshot: MvccSnapshot[applied={T|T < 7316495928088350720}]
I20260809 05:38:08.726696 18247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:08.726998 18247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:08.730849 18247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7dbdded4396b4fb2a3e7486aa746476c: Sending DeleteTablet for 3 replicas of tablet eb7ba3652fa94cf58015861e53d1fe98
I20260809 05:38:08.732177 18448 tablet_service.cc:1514] Processing DeleteTablet for tablet eb7ba3652fa94cf58015861e53d1fe98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:08 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:51132
I20260809 05:38:08.732183 18664 tablet_service.cc:1514] Processing DeleteTablet for tablet eb7ba3652fa94cf58015861e53d1fe98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:08 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:48106
I20260809 05:38:08.732247 18857 tablet_service.cc:1514] Processing DeleteTablet for tablet eb7ba3652fa94cf58015861e53d1fe98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:08 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:55208
I20260809 05:38:08.734566 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 18308
I20260809 05:38:08.742533 19029 tablet_replica.cc:330] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: stopping tablet replica
I20260809 05:38:08.744192 19029 raft_consensus.cc:2229] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:38:08.745814 19029 raft_consensus.cc:2258] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:08.751333 19027 tablet_replica.cc:330] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: stopping tablet replica
I20260809 05:38:08.751834 19031 tablet_replica.cc:330] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: stopping tablet replica
I20260809 05:38:08.752688 19031 raft_consensus.cc:2229] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:08.753319 19031 raft_consensus.cc:2258] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:08.755518 19031 ts_tablet_manager.cc:1905] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:08.755522 19029 ts_tablet_manager.cc:1905] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:08.757373 19027 raft_consensus.cc:2229] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:08.757987 19027 raft_consensus.cc:2258] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:08.764647 19027 ts_tablet_manager.cc:1905] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:08.772213 19027 ts_tablet_manager.cc:1918] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:08.775146 19029 ts_tablet_manager.cc:1918] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:08.775447 19029 log.cc:1196] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/eb7ba3652fa94cf58015861e53d1fe98
I20260809 05:38:08.775421 19031 ts_tablet_manager.cc:1918] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:08.775604 19031 log.cc:1196] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/eb7ba3652fa94cf58015861e53d1fe98
I20260809 05:38:08.776206 19029 ts_tablet_manager.cc:1939] T eb7ba3652fa94cf58015861e53d1fe98 P 39a18dccdd1e4a13b80a327431a8273c: Deleting consensus metadata
I20260809 05:38:08.776454 19031 ts_tablet_manager.cc:1939] T eb7ba3652fa94cf58015861e53d1fe98 P b8224e57775249dab7765e4a648482b3: Deleting consensus metadata
I20260809 05:38:08.778129 19027 log.cc:1196] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/eb7ba3652fa94cf58015861e53d1fe98
I20260809 05:38:08.778259 18228 catalog_manager.cc:4841] TS 39a18dccdd1e4a13b80a327431a8273c (127.30.137.194:41155): tablet eb7ba3652fa94cf58015861e53d1fe98 (table test-table [id=a5132c8c947b4081951aca6296d09981]) successfully deleted
I20260809 05:38:08.778426 18229 catalog_manager.cc:4841] TS b8224e57775249dab7765e4a648482b3 (127.30.137.195:40449): tablet eb7ba3652fa94cf58015861e53d1fe98 (table test-table [id=a5132c8c947b4081951aca6296d09981]) successfully deleted
I20260809 05:38:08.775264 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.738451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.30.137.193:34983)
0809 05:38:08.738713 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.738723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.750289 (+ 11566us) server_negotiation.cc:407] Connection header received
0809 05:38:08.750781 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.750786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.750878 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.751020 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:08.754053 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.755351 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.758163 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.758894 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.759005 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.766888 (+ 7883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:08.766903 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:08.766922 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:08.766980 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:08.771293 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.772317 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.772328 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.772331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.772508 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.773037 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.773042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.773046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.773564 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:08.773875 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.774560 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.774842 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:38:08.778880 19027 ts_tablet_manager.cc:1939] T eb7ba3652fa94cf58015861e53d1fe98 P 3064b64d810043b7b42843c72c236e1a: Deleting consensus metadata
I20260809 05:38:08.779510 18542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:08.870872 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 18308
I20260809 05:38:08.870985 18229 catalog_manager.cc:4841] TS 3064b64d810043b7b42843c72c236e1a (127.30.137.193:34983): tablet eb7ba3652fa94cf58015861e53d1fe98 (table test-table [id=a5132c8c947b4081951aca6296d09981]) successfully deleted
I20260809 05:38:08.900187 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 18552
I20260809 05:38:08.917109 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:08.901062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.30.137.194:41155)
0809 05:38:08.901283 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:08.901289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:08.901313 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:38:08.901494 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:08.901500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:08.901614 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:08.901751 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:08.903287 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.904497 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.905808 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:08.906544 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:08.906820 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:08.909631 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:08.909650 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:08.909671 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:08.909716 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:08.914020 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.914931 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.914942 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.914945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.915121 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:08.915505 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:08.915510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:08.915515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:08.915949 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:08.916288 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:08.916465 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 05:38:08.916716 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:38:08.918048 18753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:09.033658 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 18552
I20260809 05:38:09.059671 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 18766
I20260809 05:38:09.074720 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:09.060477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.30.137.195:40449)
0809 05:38:09.060668 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:09.060674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:09.060771 (+ 97us) server_negotiation.cc:407] Connection header received
0809 05:38:09.061004 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:09.061016 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:09.061083 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:09.061192 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:09.062603 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:09.063405 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:09.065032 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:09.065442 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:09.065591 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:09.068651 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:09.068667 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:09.068682 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:09.068725 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:09.071634 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:09.072430 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:09.072443 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:09.072445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:09.072600 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:09.073014 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:09.073020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:09.073023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:09.073407 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:09.073720 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:09.074024 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 05:38:09.074283 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 05:38:09.075486 18942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:09.167301 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 18766
I20260809 05:38:09.196137 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 18204
I20260809 05:38:09.197381 18298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:09.311671 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 18204
I20260809 05:38:09.343425 31271 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 05:38:09.689751 31271 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 05:38:09.690056 31271 mini_ranger.cc:66] Stopping Ranger...
I20260809 05:38:10.046761 31271 mini_ranger.cc:68] Stopped Ranger
2026-08-09 05:38:10.047 UTC [16819] LOG: received smart shutdown request
2026-08-09 05:38:10.050 UTC [16819] LOG: background worker "logical replication launcher" (PID 16831) exited with exit code 1
2026-08-09 05:38:10.050 UTC [16825] LOG: shutting down
2026-08-09 05:38:10.222 UTC [16819] LOG: database system is shut down
2026-08-09T05:38:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56022 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:38:10 dist-test-slave-3kk6 krb5kdc[19053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:38:10 dist-test-slave-3kk6 krb5kdc[19053](info): set up 2 sockets
Aug 09 05:38:10 dist-test-slave-3kk6 krb5kdc[19053](info): commencing operation
krb5kdc: starting...
W20260809 05:38:11.714964 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:38:11 dist-test-slave-3kk6 krb5kdc[19053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:38:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:38:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:38:12.024755 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43485
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43485,127.30.137.253:43569,127.30.137.252:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:12.391480 19068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:12.391858 19068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:12.392024 19068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:12.399566 19068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:12.399712 19068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:12.399807 19068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:12.399923 19068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:12.400022 19068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:12.410463 19068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43485,127.30.137.253:43569,127.30.137.252:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:12.412029 19068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:12.413503 19068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:12.426144 19075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:12.426218 19077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:12.426267 19081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:12.427513 19068 server_base.cc:1034] running on GCE node
I20260809 05:38:12.428293 19068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:12.429731 19068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:12.430938 19068 hybrid_clock.cc:648] HybridClock initialized: now 1786253892430874 us; error 53 us; skew 500 ppm
I20260809 05:38:12.431334 19068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:12 dist-test-slave-3kk6 krb5kdc[19053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:12.434856 19068 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:12.436466 19068 webserver.cc:460] Webserver started at http://127.30.137.254:40211/ using document root <none> and password file <none>
I20260809 05:38:12.437139 19068 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:12.437242 19068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:12.437513 19068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:12.440690 19068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "1eb660bc27254b59b44ecebe2670842b"
format_stamp: "Formatted at 2026-08-09 05:38:12 on dist-test-slave-3kk6"
server_key: "409660ad8289cd21742217935670abde"
server_key_iv: "44a0742f21fa18feca2cba963ba67132"
server_key_version: "encryptionkey@0"
I20260809 05:38:12.441552 19068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "1eb660bc27254b59b44ecebe2670842b"
format_stamp: "Formatted at 2026-08-09 05:38:12 on dist-test-slave-3kk6"
server_key: "409660ad8289cd21742217935670abde"
server_key_iv: "44a0742f21fa18feca2cba963ba67132"
server_key_version: "encryptionkey@0"
I20260809 05:38:12.447465 19068 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260809 05:38:12.451584 19093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:12.453095 19068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 05:38:12.453316 19068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "1eb660bc27254b59b44ecebe2670842b"
format_stamp: "Formatted at 2026-08-09 05:38:12 on dist-test-slave-3kk6"
server_key: "409660ad8289cd21742217935670abde"
server_key_iv: "44a0742f21fa18feca2cba963ba67132"
server_key_version: "encryptionkey@0"
I20260809 05:38:12.453549 19068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:12.468560 19068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:12.473075 19068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:12.473419 19068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:12.493918 19068 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:43485
I20260809 05:38:12.493949 19168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:43485 every 8 connection(s)
I20260809 05:38:12.495465 19068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:12.496840 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19068
I20260809 05:38:12.497090 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:38:12.497982 31271 external_mini_cluster.cc:1428] Setting key 6abc4a87a8a3e70b5e083db97c5a81f4
I20260809 05:38:12.500414 19169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:12.510911 19169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:38:12 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.137.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.253@KRBTEST.COM" created.
I20260809 05:38:12.535117 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.514480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43485 (local address 127.0.0.1:45202)
0809 05:38:12.515346 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.515394 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:12.517523 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:12.518386 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:12.518394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:12.519164 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:12.519520 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:12.519539 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:12.521176 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:12.521185 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:12.522294 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:12.522306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:12.523417 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:12.523424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:12.523615 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:12.524332 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:12.524367 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:12.526828 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:12.530717 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:12.530730 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:12.530739 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:12.531189 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:12.531723 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:12.531726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:12.531728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:12.531905 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:12.532964 (+ 1059us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:12.532970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:12.533166 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 05:38:12.533379 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 05:38:12.533801 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":408,"threads_started":1}
I20260809 05:38:12.535163 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.515950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.30.137.254:43485)
0809 05:38:12.516693 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:12.516703 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:12.516765 (+ 62us) server_negotiation.cc:407] Connection header received
0809 05:38:12.517917 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:12.517924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:12.518155 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:12.518374 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:12.519734 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:12.521001 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:12.522453 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:12.523163 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:12.523372 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:12.527066 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:12.527075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:12.527089 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:12.527122 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:12.530475 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:12.531358 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:12.531370 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:12.531376 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:12.531591 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:12.532026 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:12.532034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:12.532041 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:12.532817 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:12.533045 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:12.533580 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 05:38:12.534016 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":544,"thread_start_us":354,"threads_started":1}
I20260809 05:38:12.538389 19169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:38:12.539673 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.539245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50516)
0809 05:38:12.539445 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.539557 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 05:38:12.539994 19098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
I20260809 05:38:12.541760 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.541382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50522)
0809 05:38:12.541535 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.541628 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:38:12.542048 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:43569: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab.
Entry for principal kudu/127.30.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab.
I20260809 05:38:12.565768 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:43569
--webserver_interface=127.30.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab
--principal=kudu/127.30.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43485,127.30.137.253:43569,127.30.137.252:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:12.591446 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } attempt: 1
I20260809 05:38:12.592710 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.592220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50530)
0809 05:38:12.592442 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.592565 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 05:38:12.593755 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.593400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50544)
0809 05:38:12.593547 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.593623 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:38:12.593989 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:43569: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
I20260809 05:38:12.662267 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } attempt: 2
I20260809 05:38:12.663599 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.663062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50548)
0809 05:38:12.663352 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.663449 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 05:38:12.664783 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.664409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50562)
0809 05:38:12.664554 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.664643 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 05:38:12.665082 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:43569: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
I20260809 05:38:12.756373 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } attempt: 3
I20260809 05:38:12.757735 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.757226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50566)
0809 05:38:12.757436 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.757577 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 05:38:12.758882 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.758418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50574)
0809 05:38:12.758575 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.758729 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:38:12.759060 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:43569: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
I20260809 05:38:12.902334 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } attempt: 4
I20260809 05:38:12.903796 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.903102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50584)
0809 05:38:12.903461 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.903583 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260809 05:38:12.904949 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:12.904493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50592)
0809 05:38:12.904671 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:12.904792 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 05:38:12.905193 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:43569: Network error: Client connection negotiation failed: client connection to 127.30.137.253:43569: connect: Connection refused (error 111)
W20260809 05:38:12.925948 19183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:12.926265 19183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:12.926357 19183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:12.933678 19183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:12.933807 19183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:12.933878 19183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:12.933940 19183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:12.933984 19183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:12.944254 19183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43485,127.30.137.253:43569,127.30.137.252:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab
--principal=kudu/127.30.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:43569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:12.945749 19183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:12.947059 19183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:12.959616 19188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:12.959537 19193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:12.959620 19189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:12.960515 19183 server_base.cc:1034] running on GCE node
I20260809 05:38:12.961342 19183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:12.962869 19183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:12.964084 19183 hybrid_clock.cc:648] HybridClock initialized: now 1786253892964035 us; error 44 us; skew 500 ppm
I20260809 05:38:12.964453 19183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:12 dist-test-slave-3kk6 krb5kdc[19053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:12.967816 19183 init.cc:377] Logged in from keytab as kudu/127.30.137.253@KRBTEST.COM (short username kudu)
I20260809 05:38:12.969364 19183 webserver.cc:460] Webserver started at http://127.30.137.253:37321/ using document root <none> and password file <none>
I20260809 05:38:12.970140 19183 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:12.970294 19183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:12.970619 19183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:12.973805 19183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data/instance:
uuid: "6e7b82a9166d4448859cbc31d2ddf82b"
format_stamp: "Formatted at 2026-08-09 05:38:12 on dist-test-slave-3kk6"
server_key: "6335c73dcf0af3dc27b9bd3bf13e8945"
server_key_iv: "4bb00691f88237140c6ac4dc94baabbf"
server_key_version: "encryptionkey@0"
I20260809 05:38:12.974702 19183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal/instance:
uuid: "6e7b82a9166d4448859cbc31d2ddf82b"
format_stamp: "Formatted at 2026-08-09 05:38:12 on dist-test-slave-3kk6"
server_key: "6335c73dcf0af3dc27b9bd3bf13e8945"
server_key_iv: "4bb00691f88237140c6ac4dc94baabbf"
server_key_version: "encryptionkey@0"
I20260809 05:38:12.979964 19183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 05:38:12.983632 19207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:12.984951 19183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 05:38:12.985155 19183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal
uuid: "6e7b82a9166d4448859cbc31d2ddf82b"
format_stamp: "Formatted at 2026-08-09 05:38:12 on dist-test-slave-3kk6"
server_key: "6335c73dcf0af3dc27b9bd3bf13e8945"
server_key_iv: "4bb00691f88237140c6ac4dc94baabbf"
server_key_version: "encryptionkey@0"
I20260809 05:38:12.985379 19183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:13.001843 19183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:13.006145 19183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:13.006484 19183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:13.025878 19183 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.253:43569
I20260809 05:38:13.025918 19288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.253:43569 every 8 connection(s)
I20260809 05:38:13.027442 19183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
I20260809 05:38:13.031196 19289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:13.036732 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19183
I20260809 05:38:13.036928 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-1/wal/instance
I20260809 05:38:13.037398 31271 external_mini_cluster.cc:1428] Setting key 491fed17e520d9f60d939711db14a36f
I20260809 05:38:13.038444 19289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:13.062911 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.041057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.30.137.254:43485)
0809 05:38:13.041645 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.041651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.042719 (+ 1068us) server_negotiation.cc:407] Connection header received
0809 05:38:13.045686 (+ 2967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.045690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.045758 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.045858 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.048095 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.048852 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.050490 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.051263 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.051417 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.055199 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.055208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.055216 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.055277 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.059307 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.060142 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.060151 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.060157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.060312 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.060745 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.060753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.060758 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.061197 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.061406 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.062086 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.062359 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":277,"threads_started":1}
I20260809 05:38:13.063609 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.041631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43485 (local address 127.0.0.1:45214)
0809 05:38:13.042496 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.042550 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.045385 (+ 2835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.046124 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.046137 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.046998 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.047816 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.047845 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.049027 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.049035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.050363 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.050382 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.051483 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.051489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.051697 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.052434 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.052455 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.054969 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.059475 (+ 4506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.059481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.059495 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.059993 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.060432 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.060435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.060437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.060609 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.061331 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.061337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.061685 (+ 348us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.061930 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.062336 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":318,"threads_started":1}
I20260809 05:38:13.065411 19289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.137.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.252@KRBTEST.COM" created.
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:38:13.082628 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.066236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50604)
0809 05:38:13.066439 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.066461 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.066568 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.067754 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.067759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.068209 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.068345 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.068353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.069811 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.069815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.070868 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.070880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.072164 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.072168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.072359 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.073053 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.073070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.075139 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.079676 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.079680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.079683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.080076 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.080537 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.080540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.080542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.080678 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.081459 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.081466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.081631 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.081773 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.082069 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":1}
I20260809 05:38:13.083200 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.066557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.30.137.253:43569)
0809 05:38:13.067306 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.067312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.067335 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:38:13.067377 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.067382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.067608 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.067764 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.068546 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.069635 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.071020 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.071993 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.072156 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.075380 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.075397 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.075418 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.075460 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.079508 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.080217 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.080229 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.080234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.080421 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.080784 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.080792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.080798 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.081320 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.081547 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.081936 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.082228 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":595,"thread_start_us":448,"threads_started":1}
I20260809 05:38:13.085981 19289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:38:13.087365 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.086874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40666)
0809 05:38:13.087087 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.087201 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 05:38:13.087685 19208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.089440 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.089036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40678)
0809 05:38:13.089177 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.089304 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 05:38:13.089690 19289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab.
Entry for principal kudu/127.30.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab.
I20260809 05:38:13.107658 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:37847
--webserver_interface=127.30.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab
--principal=kudu/127.30.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43485,127.30.137.253:43569,127.30.137.252:37847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:13.138994 19289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 1
I20260809 05:38:13.140285 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.139805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40680)
0809 05:38:13.140007 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.140123 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 05:38:13.141376 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.141010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40694)
0809 05:38:13.141192 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.141264 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 05:38:13.141595 19289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.204517 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } attempt: 5
I20260809 05:38:13.209813 19289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 2
I20260809 05:38:13.211035 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.210402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40698)
0809 05:38:13.210712 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.210831 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260809 05:38:13.212180 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.211787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40708)
0809 05:38:13.211965 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.212052 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 05:38:13.212507 19289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:38:13.221113 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.205332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50614)
0809 05:38:13.205549 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.205572 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.205717 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.206110 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.206115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.206686 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.206949 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.206963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.207952 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.207958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.209331 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.209348 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.210527 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.210532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.210819 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.211629 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.211656 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.214260 (+ 2604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.218468 (+ 4208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.218472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.218475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.219024 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.219456 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.219459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.219462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.219601 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.220210 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.220213 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.220420 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.220534 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.220769 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 05:38:13.221236 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.205450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.30.137.253:43569)
0809 05:38:13.205635 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.205639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.205660 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:38:13.205859 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.205863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.205945 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.206033 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.207113 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.207805 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.209532 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.210353 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.210492 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.214468 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.214475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.214480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.214521 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.218290 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.219159 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.219166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.219169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.219337 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.219723 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.219728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.219731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.220093 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.220260 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.220694 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.220935 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 05:38:13.222407 19169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:38:13.223559 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.223167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40724)
0809 05:38:13.223348 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.223434 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 05:38:13.224565 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.224266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40730)
0809 05:38:13.224389 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.224467 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 05:38:13.224843 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.276105 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 1
I20260809 05:38:13.277376 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.276905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40738)
0809 05:38:13.277103 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.277199 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:38:13.278420 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.278071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40744)
0809 05:38:13.278212 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.278298 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 05:38:13.278690 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.303745 19289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 3
I20260809 05:38:13.305095 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.304609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40752)
0809 05:38:13.304817 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.304925 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 05:38:13.306186 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.305761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40764)
0809 05:38:13.305992 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.306069 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260809 05:38:13.306412 19289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.346911 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 2
I20260809 05:38:13.348273 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.347751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40776)
0809 05:38:13.347995 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.348100 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 05:38:13.349306 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.348959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40780)
0809 05:38:13.349103 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.349180 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 05:38:13.349526 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.449762 19289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 4
I20260809 05:38:13.451077 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.450594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40794)
0809 05:38:13.450841 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.450922 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 05:38:13.452131 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.451721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40808)
0809 05:38:13.451908 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.452001 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 05:38:13.452344 19289 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
I20260809 05:38:13.455717 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 3
I20260809 05:38:13.456707 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.456290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40818)
0809 05:38:13.456470 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.456561 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 05:38:13.457832 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.457473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40824)
0809 05:38:13.457651 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.457726 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 05:38:13.458057 19169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:37847: Network error: Client connection negotiation failed: client connection to 127.30.137.252:37847: connect: Connection refused (error 111)
W20260809 05:38:13.467499 19298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:13.467854 19298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:13.468011 19298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:13.475483 19298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:13.475641 19298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:13.475749 19298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:13.475855 19298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:13.475934 19298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:13.486418 19298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43485,127.30.137.253:43569,127.30.137.252:37847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab
--principal=kudu/127.30.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:37847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:13.487823 19298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:13.489243 19298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:13.502298 19310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:13.502566 19306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:13.502804 19312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:13.503132 19298 server_base.cc:1034] running on GCE node
I20260809 05:38:13.504120 19298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:13.505879 19298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:13.507117 19298 hybrid_clock.cc:648] HybridClock initialized: now 1786253893507052 us; error 58 us; skew 500 ppm
I20260809 05:38:13.507504 19298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253893, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:13.510784 19298 init.cc:377] Logged in from keytab as kudu/127.30.137.252@KRBTEST.COM (short username kudu)
I20260809 05:38:13.512215 19298 webserver.cc:460] Webserver started at http://127.30.137.252:43823/ using document root <none> and password file <none>
I20260809 05:38:13.512871 19298 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:13.512970 19298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:13.513227 19298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:13.516413 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data/instance:
uuid: "f5b0591de8244f3db3f1dd1d60809804"
format_stamp: "Formatted at 2026-08-09 05:38:13 on dist-test-slave-3kk6"
server_key: "dd2908f913ee767c950c966e359be415"
server_key_iv: "85fee28ddb41ae05d46849fc56b1e99a"
server_key_version: "encryptionkey@0"
I20260809 05:38:13.517203 19298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal/instance:
uuid: "f5b0591de8244f3db3f1dd1d60809804"
format_stamp: "Formatted at 2026-08-09 05:38:13 on dist-test-slave-3kk6"
server_key: "dd2908f913ee767c950c966e359be415"
server_key_iv: "85fee28ddb41ae05d46849fc56b1e99a"
server_key_version: "encryptionkey@0"
I20260809 05:38:13.522339 19298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:38:13.525964 19322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:13.527343 19298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:13.527519 19298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal
uuid: "f5b0591de8244f3db3f1dd1d60809804"
format_stamp: "Formatted at 2026-08-09 05:38:13 on dist-test-slave-3kk6"
server_key: "dd2908f913ee767c950c966e359be415"
server_key_iv: "85fee28ddb41ae05d46849fc56b1e99a"
server_key_version: "encryptionkey@0"
I20260809 05:38:13.527745 19298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:13.541390 19298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:13.545840 19298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:13.546172 19298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:13.566140 19298 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.252:37847
I20260809 05:38:13.566152 19386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.252:37847 every 8 connection(s)
I20260809 05:38:13.567713 19298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
I20260809 05:38:13.571593 19387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:13.577030 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19298
I20260809 05:38:13.577243 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786253675866932-31271-0/minicluster-data/master-2/wal/instance
I20260809 05:38:13.577836 31271 external_mini_cluster.cc:1428] Setting key f70322d339c45c56bf26bc441fb1ce3f
I20260809 05:38:13.579493 19387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253893, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
I20260809 05:38:13.600422 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.582489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.30.137.254:43485)
0809 05:38:13.583183 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.583187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.583528 (+ 341us) server_negotiation.cc:407] Connection header received
0809 05:38:13.584235 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.584241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.584339 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.584476 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.585904 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.586751 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.588671 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.589344 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.589508 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.593781 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.593786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.593790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.593832 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.596909 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.597774 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.597779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.597784 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.597938 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.598450 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.598457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.598460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.598826 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.599001 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.599755 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.600004 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":488,"threads_started":1}
I20260809 05:38:13.601680 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.580190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.30.137.252:37847)
0809 05:38:13.581062 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.581074 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.581118 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:38:13.583967 (+ 2849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.583988 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.584256 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.584468 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.586063 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.587702 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.589538 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.590409 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.590552 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.595085 (+ 4533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.595094 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.595107 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.595136 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.597907 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.598782 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.598791 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.598794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.598928 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.599346 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.599353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.599360 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.599723 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.600044 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.600403 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.600677 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":420,"threads_started":1}
I20260809 05:38:13.601720 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.582451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:43485 (local address 127.0.0.1:45226)
0809 05:38:13.583371 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.583419 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.583979 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.584581 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.584593 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.585305 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.585687 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.585705 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.586914 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.586920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.588492 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.588510 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.589631 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.589638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.589957 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.590909 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.590939 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.593563 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.597067 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.597078 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.597091 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.597627 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.598060 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.598065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.598069 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.598317 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.598980 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.598988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.599348 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.599663 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.600199 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":358,"threads_started":1}
I20260809 05:38:13.604393 19387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253893, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:13.618781 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.605230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:43569 (local address 127.0.0.1:50624)
0809 05:38:13.605421 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.605443 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.605544 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.605905 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.605910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.606296 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.606449 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.606457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.607508 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.607514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.608407 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.608419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.609134 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.609140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.609367 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.610024 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.610042 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.612321 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.616101 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.616105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.616108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.616497 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.616916 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.616919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.616922 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.617064 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.617717 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.617722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.617881 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.618051 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.618362 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 05:38:13.618809 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.605235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.30.137.253:43569)
0809 05:38:13.605453 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.605459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.605498 (+ 39us) server_negotiation.cc:407] Connection header received
0809 05:38:13.605685 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.605690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.605765 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.605868 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.606606 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.607355 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.608551 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.608978 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.609112 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.612543 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.612548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.612551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.612586 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.615949 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.616628 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.616633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.616636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.616796 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.617179 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.617186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.617189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.617585 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.617737 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.618147 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.618360 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 05:38:13.619894 19387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:38:13.620733 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.605242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.30.137.254:43485)
0809 05:38:13.605735 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.605739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.605753 (+ 14us) server_negotiation.cc:407] Connection header received
0809 05:38:13.605794 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.605797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.605869 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.605980 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.607268 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.607968 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.609245 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.609675 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.609816 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.614075 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.614082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.614087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.614133 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.617452 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.618222 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.618228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.618233 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.618367 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.619086 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.619091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.619095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.619434 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.619730 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.620125 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.620397 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390}
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253893, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:38:13.633993 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.620779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.30.137.252:37847)
0809 05:38:13.620962 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.620969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.621185 (+ 216us) server_negotiation.cc:407] Connection header received
0809 05:38:13.621294 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.621299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.621373 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.621450 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.622354 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.623425 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.624571 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.625054 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.625149 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.628493 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.628498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.628502 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.628530 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.631358 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.632006 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.632011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.632014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.632163 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.632522 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.632529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.632532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.632931 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.633088 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.633426 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.633671 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 05:38:13.634423 19391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.620770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40844)
0809 05:38:13.621079 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.621097 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.621210 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.621514 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.621519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.622032 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.622226 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.622235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.623563 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.623567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.624424 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.624434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.625204 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.625210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.625404 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.626062 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.626079 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.628277 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.631488 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.631495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.631499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.631887 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.632258 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.632265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.632271 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.632416 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.633039 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.633044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.633195 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.633349 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.633948 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":223}
I20260809 05:38:13.635278 19169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 4
I20260809 05:38:13.638173 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.622843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.30.137.253:43569)
0809 05:38:13.623012 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.623016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.623031 (+ 15us) server_negotiation.cc:407] Connection header received
0809 05:38:13.623177 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.623184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.623233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.623315 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.624668 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.625651 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.627084 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.627571 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.627699 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.632181 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.632197 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.632202 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.632241 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.635282 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.636158 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.636163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.636168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.636352 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.636788 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.636796 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.636798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.637064 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.637360 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.637644 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.637869 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:38:13.640568 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: Bootstrap starting.
I20260809 05:38:13.643250 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:38:13.644433 19387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:13.645498 19387 log.cc:824] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:13.652846 19387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: No bootstrap required, opened a new log
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
I20260809 05:38:13.666180 19387 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.666913 19387 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:13.667114 19387 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b0591de8244f3db3f1dd1d60809804, State: Initialized, Role: FOLLOWER
I20260809 05:38:13.667747 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.635931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40856)
0809 05:38:13.636112 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.636130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.636227 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.646281 (+ 10054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.646290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.647073 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.647407 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.647426 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.649503 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.649508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.651091 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.651112 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.652790 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.652803 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.653091 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.653883 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.653915 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.657435 (+ 3520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.662669 (+ 5234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.662678 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.662685 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.663414 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.663957 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.663965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.663971 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.664185 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.665135 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.665143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.665335 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.665511 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.667359 (+ 1848us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 05:38:13.667851 19387 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.666920 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.635931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.30.137.252:37847)
0809 05:38:13.645815 (+ 9884us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.645822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.645844 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:38:13.645914 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.645922 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.646019 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.647619 (+ 1600us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.647813 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.649308 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.651284 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.652351 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.652505 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.657666 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.657676 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.657684 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.657733 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.662440 (+ 4707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.663583 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.663592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.663598 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.663794 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.664332 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.664340 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.664346 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.664842 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.665033 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.666073 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.666366 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9734}
I20260809 05:38:13.671262 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.649148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.30.137.254:43485)
0809 05:38:13.649774 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.649780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.649796 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:38:13.649842 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.649846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.649920 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.650032 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.652342 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.653819 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.656220 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.656943 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.657045 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.662499 (+ 5454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.662506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.662511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.662565 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.666358 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.668007 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.668012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.668016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.668161 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.668734 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.668743 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.668748 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.669221 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.669575 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.670331 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.670667 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508}
I20260809 05:38:13.675516 19169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: Bootstrap starting.
I20260809 05:38:13.679885 19414 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.682126 19169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:13.682372 19414 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.681242 19387 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:13.686268 19169 log.cc:824] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:13.696180 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.650428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.30.137.253:43569)
0809 05:38:13.650945 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.650951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.674626 (+ 23675us) server_negotiation.cc:407] Connection header received
0809 05:38:13.674982 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.674991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.675105 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.675196 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.677795 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.679318 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.681294 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.682058 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.682177 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.686763 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.686774 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.686780 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.686843 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.691714 (+ 4871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.692994 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.693004 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.693010 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.693235 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.693888 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.693895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.693901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.694410 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.694749 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.695482 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.695760 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260809 05:38:13.700929 19169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: No bootstrap required, opened a new log
I20260809 05:38:13.703522 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.650594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.30.137.252:37847)
0809 05:38:13.668584 (+ 17990us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.668589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.682164 (+ 13575us) server_negotiation.cc:407] Connection header received
0809 05:38:13.682434 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.682440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.682522 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.682602 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.685717 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.686877 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.689464 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.690421 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.690547 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.695171 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.695181 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.695189 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.695242 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.699488 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.700226 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.700232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.700235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.700445 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.701021 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.701026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.701118 (+ 92us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.701412 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.701700 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.702746 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.703049 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17890,"thread_start_us":30866,"threads_started":1}
I20260809 05:38:13.714709 19169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.715176 19169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 05:38:13.715186 19431 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 05:38:13.715304 19169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eb660bc27254b59b44ecebe2670842b, State: Initialized, Role: FOLLOWER
W20260809 05:38:13.715339 19431 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:38:13.715982 19169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.719161 19436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.719609 19436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.720580 19169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:13.712764 19298 master_runner.cc:186] Error getting master registration for 127.30.137.254:43485: OK, instance_id { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" instance_seqno: 1786253892488178 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:38:13.725591 19298 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 05:38:13.726744 19106 tablet.cc:2369] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:13.733711 19447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:38:13.733937 19447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:38:13.730581 19068 master_runner.cc:186] Error getting master registration for 127.30.137.253:43569: OK, instance_id { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" instance_seqno: 1786253893020129 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:38:13.742117 19068 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:38:13.751582 19289 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } attempt: 5
Aug 09 05:38:13 dist-test-slave-3kk6 krb5kdc[19053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253892, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
I20260809 05:38:13.773936 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.752375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:37847 (local address 127.0.0.1:40866)
0809 05:38:13.752776 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:13.752798 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:13.752929 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:13.753866 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:13.753876 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:13.754517 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:13.754787 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.754802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.756366 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.756371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.757812 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:13.757829 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.759005 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.759011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:13.759227 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.759992 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:13.760018 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:13.762972 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:13.770767 (+ 7795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.770773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.770777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.771346 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.771842 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:13.771847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:13.771858 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:13.772048 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:13.772732 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:13.772738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:13.772956 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 05:38:13.773122 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 05:38:13.773450 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260809 05:38:13.773994 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:13.753161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.30.137.252:37847)
0809 05:38:13.753356 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:13.753366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:13.753385 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:38:13.753444 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:13.753450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:13.753527 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:13.753623 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:13.755006 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.756159 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.757988 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:13.758840 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:13.759002 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:13.763212 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:13.763220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:13.763225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:13.763266 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:13.770597 (+ 7331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.771490 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.771497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.771500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.771704 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:13.772182 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:13.772187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:13.772191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:13.772585 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:13.772761 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:13.773278 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 05:38:13.773562 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:38:13.780664 19289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: Bootstrap starting.
I20260809 05:38:13.785269 19289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:13.786828 19289 log.cc:824] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:13.787348 19414 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:38:13.787688 19414 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.789439 19414 leader_election.cc:288] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1eb660bc27254b59b44ecebe2670842b (127.30.137.254:43485), 6e7b82a9166d4448859cbc31d2ddf82b (127.30.137.253:43569)
I20260809 05:38:13.790729 19133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5b0591de8244f3db3f1dd1d60809804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eb660bc27254b59b44ecebe2670842b" is_pre_election: true
I20260809 05:38:13.791435 19133 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b0591de8244f3db3f1dd1d60809804 in term 0.
I20260809 05:38:13.792218 19326 leader_election.cc:302] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eb660bc27254b59b44ecebe2670842b, f5b0591de8244f3db3f1dd1d60809804; no voters:
I20260809 05:38:13.792755 19414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:38:13.792922 19414 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:38:13.793048 19414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:13.796693 19414 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.797840 19133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5b0591de8244f3db3f1dd1d60809804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eb660bc27254b59b44ecebe2670842b"
I20260809 05:38:13.798063 19133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:13.801754 19133 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b0591de8244f3db3f1dd1d60809804 in term 1.
W20260809 05:38:13.802400 19332 tablet.cc:2369] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:38:13.803375 19326 leader_election.cc:302] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eb660bc27254b59b44ecebe2670842b, f5b0591de8244f3db3f1dd1d60809804; no voters:
I20260809 05:38:13.805284 19289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: No bootstrap required, opened a new log
I20260809 05:38:13.806929 19414 leader_election.cc:288] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [CANDIDATE]: Term 1 election: Requested vote from peers 1eb660bc27254b59b44ecebe2670842b (127.30.137.254:43485), 6e7b82a9166d4448859cbc31d2ddf82b (127.30.137.253:43569)
I20260809 05:38:13.807739 19414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:13.808406 19414 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [term 1 LEADER]: Becoming Leader. State: Replica: f5b0591de8244f3db3f1dd1d60809804, State: Running, Role: LEADER
I20260809 05:38:13.810783 19414 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.828325 19289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.829248 19289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:13.829545 19289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7b82a9166d4448859cbc31d2ddf82b, State: Initialized, Role: FOLLOWER
I20260809 05:38:13.830914 19289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } }
I20260809 05:38:13.831735 19452 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5b0591de8244f3db3f1dd1d60809804. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.832298 19452 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:13.840395 19455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:13.843052 19455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:13.844712 19255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5b0591de8244f3db3f1dd1d60809804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" is_pre_election: true
I20260809 05:38:13.845417 19255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b0591de8244f3db3f1dd1d60809804 in term 0.
I20260809 05:38:13.847012 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f5b0591de8244f3db3f1dd1d60809804" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7b82a9166d4448859cbc31d2ddf82b"
I20260809 05:38:13.847302 19254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:13.852770 19254 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b0591de8244f3db3f1dd1d60809804 in term 1.
I20260809 05:38:13.853642 19289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:13.855111 19454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.856338 19454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.863165 19254 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [term 1 FOLLOWER]: Refusing update from remote peer f5b0591de8244f3db3f1dd1d60809804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:13.864431 19452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:13.873721 19133 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [term 1 FOLLOWER]: Refusing update from remote peer f5b0591de8244f3db3f1dd1d60809804: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:13.874745 19452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:13.880949 19454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5b0591de8244f3db3f1dd1d60809804. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.881278 19454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.891026 19454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.891341 19454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.894137 19414 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.894512 19414 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:13.897485 19455 catalog_manager.cc:1347] Generated new cluster ID: 1b7ce30902c643ec884c693827595dfa
I20260809 05:38:13.897599 19455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:13.898910 19436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5b0591de8244f3db3f1dd1d60809804. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.899257 19436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.902594 19452 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.902906 19452 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:13.912875 19436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5b0591de8244f3db3f1dd1d60809804" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eb660bc27254b59b44ecebe2670842b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43485 } } peers { permanent_uuid: "6e7b82a9166d4448859cbc31d2ddf82b" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 43569 } } peers { permanent_uuid: "f5b0591de8244f3db3f1dd1d60809804" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 37847 } } }
I20260809 05:38:13.913177 19436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:38:13.940609 19455 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:38:13.946929 19455 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:13.973855 19455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5b0591de8244f3db3f1dd1d60809804: Generated new TSK 0
I20260809 05:38:13.975244 19455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:38:13.989595 19483 catalog_manager.cc:1259] Loaded cluster ID: 1b7ce30902c643ec884c693827595dfa
I20260809 05:38:13.989822 19483 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: loading cluster ID for follower catalog manager: success
I20260809 05:38:14.017220 19483 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: acquiring CA information for follower catalog manager: success
I20260809 05:38:14.024484 19483 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e7b82a9166d4448859cbc31d2ddf82b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:38:14.737704 19447 catalog_manager.cc:1259] Loaded cluster ID: 1b7ce30902c643ec884c693827595dfa
I20260809 05:38:14.737836 19447 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: loading cluster ID for follower catalog manager: success
I20260809 05:38:14.740768 19447 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: acquiring CA information for follower catalog manager: success
I20260809 05:38:14.742775 19447 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1eb660bc27254b59b44ecebe2670842b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:38:15.138988 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19068
I20260809 05:38:15.140362 19163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:15.252646 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19068
I20260809 05:38:15.283955 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19183
I20260809 05:38:15.285053 19284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:15.393175 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19183
I20260809 05:38:15.421903 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19298
I20260809 05:38:15.423141 19382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:15.532462 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19298
2026-08-09T05:38:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5176 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:38:15 dist-test-slave-3kk6 krb5kdc[19498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:38:15 dist-test-slave-3kk6 krb5kdc[19498](info): set up 2 sockets
Aug 09 05:38:15 dist-test-slave-3kk6 krb5kdc[19498](info): commencing operation
krb5kdc: starting...
W20260809 05:38:16.870368 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:38:17 dist-test-slave-3kk6 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:38:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:38:17.168633 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:46389
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:43247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:17.517655 19511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:17.518046 19511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:17.518210 19511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:17.525636 19511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:17.525815 19511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:17.525925 19511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:17.526054 19511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:17.526134 19511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:17.536207 19511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:17.537662 19511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:17.539080 19511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:17.551950 19519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:17.552057 19521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:17.552093 19517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:17.553026 19511 server_base.cc:1034] running on GCE node
I20260809 05:38:17.553930 19511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:17.555573 19511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:17.556927 19511 hybrid_clock.cc:648] HybridClock initialized: now 1786253897556872 us; error 58 us; skew 500 ppm
I20260809 05:38:17.557324 19511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:17 dist-test-slave-3kk6 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253897, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:17.565187 19511 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:17.567041 19511 webserver.cc:460] Webserver started at http://127.30.137.254:33281/ using document root <none> and password file <none>
I20260809 05:38:17.567776 19511 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:17.567937 19511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:17.568275 19511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:17.571956 19511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba"
format_stamp: "Formatted at 2026-08-09 05:38:17 on dist-test-slave-3kk6"
server_key: "4299dfec5121ac17d1f57e31b649c9b9"
server_key_iv: "84c06fa502dbffa52ab4b116c3805cf0"
server_key_version: "encryptionkey@0"
I20260809 05:38:17.572765 19511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba"
format_stamp: "Formatted at 2026-08-09 05:38:17 on dist-test-slave-3kk6"
server_key: "4299dfec5121ac17d1f57e31b649c9b9"
server_key_iv: "84c06fa502dbffa52ab4b116c3805cf0"
server_key_version: "encryptionkey@0"
I20260809 05:38:17.578436 19511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:38:17.582137 19531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:17.583596 19511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:17.583793 19511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba"
format_stamp: "Formatted at 2026-08-09 05:38:17 on dist-test-slave-3kk6"
server_key: "4299dfec5121ac17d1f57e31b649c9b9"
server_key_iv: "84c06fa502dbffa52ab4b116c3805cf0"
server_key_version: "encryptionkey@0"
I20260809 05:38:17.584017 19511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:17.610836 19511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:17.615089 19511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:17.615427 19511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:17.635618 19511 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:46389
I20260809 05:38:17.635637 19608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:46389 every 8 connection(s)
I20260809 05:38:17.637113 19511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:17.639071 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19511
I20260809 05:38:17.639314 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:38:17.640228 31271 external_mini_cluster.cc:1428] Setting key 68b3f5c67b0b863dfbdf541b9c63e393
I20260809 05:38:17.642186 19609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:38:17 dist-test-slave-3kk6 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:17.657343 19609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba: Bootstrap starting.
I20260809 05:38:17.660833 19609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:17.661988 19609 log.cc:824] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:17.665215 19609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba: No bootstrap required, opened a new log
I20260809 05:38:17.666107 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:17.642370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.30.137.254:46389)
0809 05:38:17.643250 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:17.643262 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:17.643312 (+ 50us) server_negotiation.cc:407] Connection header received
0809 05:38:17.645032 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:17.645054 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:17.645459 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:17.645820 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:17.647897 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:17.649408 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:17.651390 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:17.652095 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:17.652243 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:17.657859 (+ 5616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:17.657869 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:17.657885 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:17.657931 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:17.661395 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:17.662416 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:17.662424 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:17.662430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:17.662612 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:17.663129 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:17.663141 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:17.663143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:17.663706 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:17.664014 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:17.664548 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 05:38:17.665011 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":421,"threads_started":1}
I20260809 05:38:17.670290 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:38:17.673870 19609 raft_consensus.cc:348] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46389 } }
I20260809 05:38:17.674302 19609 raft_consensus.cc:374] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:17.674417 19609 raft_consensus.cc:729] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06e5a85a51b454a91ab5bcb1c29a9ba, State: Initialized, Role: FOLLOWER
I20260809 05:38:17.675082 19609 consensus_queue.cc:260] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46389 } }
I20260809 05:38:17.675344 19609 raft_consensus.cc:388] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:17.675503 19609 raft_consensus.cc:482] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:17.675696 19609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:17.678059 19609 raft_consensus.cc:504] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46389 } }
I20260809 05:38:17.678609 19609 leader_election.cc:302] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b06e5a85a51b454a91ab5bcb1c29a9ba; no voters:
I20260809 05:38:17.679327 19609 leader_election.cc:288] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:38:17.679493 19615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:17.680291 19615 raft_consensus.cc:686] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [term 1 LEADER]: Becoming Leader. State: Replica: b06e5a85a51b454a91ab5bcb1c29a9ba, State: Running, Role: LEADER
I20260809 05:38:17.680882 19615 consensus_queue.cc:237] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46389 } }
I20260809 05:38:17.681667 19609 sys_catalog.cc:564] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:17.683924 19617 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader b06e5a85a51b454a91ab5bcb1c29a9ba. Latest consensus state: current_term: 1 leader_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46389 } } }
I20260809 05:38:17.684428 19617 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:17.686062 19627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:17.685967 19616 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06e5a85a51b454a91ab5bcb1c29a9ba" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 46389 } } }
I20260809 05:38:17.686323 19616 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:17.691953 19627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:17.701450 19627 catalog_manager.cc:1347] Generated new cluster ID: 4d1e178ec9e945d488d02004bc8524b8
I20260809 05:38:17.701587 19627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:17.727351 19627 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:38:17.729322 19627 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:17.742070 19627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b06e5a85a51b454a91ab5bcb1c29a9ba: Generated new TSK 0
I20260809 05:38:17.743310 19627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:38:17.783751 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19511
I20260809 05:38:17.785115 19600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:17.881773 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19511
2026-08-09T05:38:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2347 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:38:17 dist-test-slave-3kk6 krb5kdc[19646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:38:17 dist-test-slave-3kk6 krb5kdc[19646](info): set up 2 sockets
Aug 09 05:38:17 dist-test-slave-3kk6 krb5kdc[19646](info): commencing operation
krb5kdc: starting...
W20260809 05:38:19.213104 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:38:19 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:38:19.527105 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:38275
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:38275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:19.882790 19667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:19.883106 19667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:19.883279 19667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:19.890640 19667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:19.890790 19667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:19.890918 19667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:19.891019 19667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:19.891106 19667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:19.901275 19667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:38275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:38275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:19.902801 19667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:19.904235 19667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:19.916698 19674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:19.916744 19675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:19.916950 19677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:19.917834 19667 server_base.cc:1034] running on GCE node
I20260809 05:38:19.918617 19667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:19.920192 19667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:19.921413 19667 hybrid_clock.cc:648] HybridClock initialized: now 1786253899921371 us; error 40 us; skew 500 ppm
I20260809 05:38:19.921813 19667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:19 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:19.925170 19667 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:19.926859 19667 webserver.cc:460] Webserver started at http://127.30.137.254:43599/ using document root <none> and password file <none>
I20260809 05:38:19.927623 19667 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:19.927786 19667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:19.928114 19667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:19.931337 19667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "d111d9a125d249f9b712fe1819fb1f38"
format_stamp: "Formatted at 2026-08-09 05:38:19 on dist-test-slave-3kk6"
server_key: "d41f6b7d6d7d147f5d70d13e98f1d3bf"
server_key_iv: "1109dcd15c0a135ca653c47cea921f1a"
server_key_version: "encryptionkey@0"
I20260809 05:38:19.932194 19667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "d111d9a125d249f9b712fe1819fb1f38"
format_stamp: "Formatted at 2026-08-09 05:38:19 on dist-test-slave-3kk6"
server_key: "d41f6b7d6d7d147f5d70d13e98f1d3bf"
server_key_iv: "1109dcd15c0a135ca653c47cea921f1a"
server_key_version: "encryptionkey@0"
I20260809 05:38:19.937878 19667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:38:19.941735 19685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:19.943280 19667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:19.943482 19667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "d111d9a125d249f9b712fe1819fb1f38"
format_stamp: "Formatted at 2026-08-09 05:38:19 on dist-test-slave-3kk6"
server_key: "d41f6b7d6d7d147f5d70d13e98f1d3bf"
server_key_iv: "1109dcd15c0a135ca653c47cea921f1a"
server_key_version: "encryptionkey@0"
I20260809 05:38:19.943715 19667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:19.968091 19667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:19.972707 19667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:19.973055 19667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:19.992897 19667 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:38275
I20260809 05:38:19.992938 19754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:38275 every 8 connection(s)
I20260809 05:38:19.994441 19667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:19.998492 19755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:19.998828 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19667
I20260809 05:38:19.999081 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:38:19.999835 31271 external_mini_cluster.cc:1428] Setting key fe35415747573e55775afb14b2dbf995
I20260809 05:38:20.010910 19755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap starting.
I20260809 05:38:20.014907 19755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:20.016175 19755 log.cc:824] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:20.020179 19755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: No bootstrap required, opened a new log
Aug 09 05:38:20 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:20.031956 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:20.002418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.30.137.254:38275)
0809 05:38:20.007309 (+ 4891us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:20.007321 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:20.007377 (+ 56us) server_negotiation.cc:407] Connection header received
0809 05:38:20.009269 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:20.009297 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:20.009727 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:20.010105 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:20.012232 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:20.013611 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:20.015659 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:20.016472 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:20.016621 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:20.022178 (+ 5557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:20.022195 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:20.022218 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:20.022274 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:20.026808 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:20.027630 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:20.027645 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:20.027656 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:20.027900 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:20.028305 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:20.028314 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:20.028318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:20.029146 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:20.029493 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:20.029907 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 05:38:20.030428 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4627,"thread_start_us":4324,"threads_started":1}
I20260809 05:38:20.031703 19755 raft_consensus.cc:348] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:20.032230 19755 raft_consensus.cc:374] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:20.032369 19755 raft_consensus.cc:729] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d111d9a125d249f9b712fe1819fb1f38, State: Initialized, Role: FOLLOWER
I20260809 05:38:20.033083 19755 consensus_queue.cc:260] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:20.033401 19755 raft_consensus.cc:388] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:20.033558 19755 raft_consensus.cc:482] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:20.033766 19755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:20.036571 19755 raft_consensus.cc:504] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:20.037091 19755 leader_election.cc:302] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d111d9a125d249f9b712fe1819fb1f38; no voters:
I20260809 05:38:20.037726 19755 leader_election.cc:288] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:38:20.037889 19762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:20.038586 19762 raft_consensus.cc:686] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 LEADER]: Becoming Leader. State: Replica: d111d9a125d249f9b712fe1819fb1f38, State: Running, Role: LEADER
I20260809 05:38:20.039240 19762 consensus_queue.cc:237] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:20.040016 19755 sys_catalog.cc:564] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:20.042271 19764 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d111d9a125d249f9b712fe1819fb1f38. Latest consensus state: current_term: 1 leader_uuid: "d111d9a125d249f9b712fe1819fb1f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } } }
I20260809 05:38:20.042213 19763 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d111d9a125d249f9b712fe1819fb1f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } } }
I20260809 05:38:20.042796 19764 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:20.042816 19763 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:20.046530 19775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:20.052496 19775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:20.062547 19775 catalog_manager.cc:1347] Generated new cluster ID: 9fce34ef1a524b16bd7d1715151b8761
I20260809 05:38:20.062716 19775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:20.079589 19775 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:38:20.081032 19775 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:20.093595 19775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Generated new TSK 0
I20260809 05:38:20.094861 19775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:38:20.166988 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:38275
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:20.526643 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:20.527112 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:20.527225 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:20.535135 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:20.535269 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:20.535386 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:38:20.545948 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:20.547505 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:20.548919 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:20.562783 19812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:20.562963 19810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:20.562901 19815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:20.563804 19798 server_base.cc:1034] running on GCE node
I20260809 05:38:20.564462 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:20.565459 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:20.566733 19798 hybrid_clock.cc:648] HybridClock initialized: now 1786253900566674 us; error 51 us; skew 500 ppm
I20260809 05:38:20.567065 19798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:20 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:20.570552 19798 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:38:20.572058 19798 webserver.cc:460] Webserver started at http://127.30.137.193:45005/ using document root <none> and password file <none>
I20260809 05:38:20.572754 19798 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:20.572916 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:20.573230 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:20.576318 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "12342fed55b44c7883d4b64bb5f554d7"
format_stamp: "Formatted at 2026-08-09 05:38:20 on dist-test-slave-3kk6"
server_key: "72b5098cb84ddc3f3734d27da9c71402"
server_key_iv: "afe68270e468f5944c1849d1aa8e29fa"
server_key_version: "encryptionkey@0"
I20260809 05:38:20.577139 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "12342fed55b44c7883d4b64bb5f554d7"
format_stamp: "Formatted at 2026-08-09 05:38:20 on dist-test-slave-3kk6"
server_key: "72b5098cb84ddc3f3734d27da9c71402"
server_key_iv: "afe68270e468f5944c1849d1aa8e29fa"
server_key_version: "encryptionkey@0"
I20260809 05:38:20.582608 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:38:20.586191 19823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:20.587574 19798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:20.587760 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "12342fed55b44c7883d4b64bb5f554d7"
format_stamp: "Formatted at 2026-08-09 05:38:20 on dist-test-slave-3kk6"
server_key: "72b5098cb84ddc3f3734d27da9c71402"
server_key_iv: "afe68270e468f5944c1849d1aa8e29fa"
server_key_version: "encryptionkey@0"
I20260809 05:38:20.587980 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:20.610402 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:20.615108 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:20.615439 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:20.616503 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:20.617889 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:20.617969 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:20.618124 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:20.618191 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:20.653618 19798 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44221
I20260809 05:38:20.653669 19985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44221 every 8 connection(s)
I20260809 05:38:20.655037 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:38:20.658790 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19798
I20260809 05:38:20.659039 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:38:20.659847 31271 external_mini_cluster.cc:1428] Setting key 589f23a69267f6151d1ef85783ed3e28
Aug 09 05:38:20 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:20.677596 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:20.657832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:54041 (local address 127.30.137.254:38275)
0809 05:38:20.658350 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:20.658359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:20.659385 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 05:38:20.661431 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:20.661435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:20.661501 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:20.661599 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:20.663210 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:20.664405 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:20.665991 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:20.666715 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:20.666863 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:20.670720 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:20.670730 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:20.670738 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:20.670786 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:20.674401 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:20.675206 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:20.675212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:20.675216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:20.675355 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:20.675824 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:20.675831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:20.675838 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:20.676114 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:20.676274 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:20.676996 (+ 722us) server_negotiation.cc:299] Negotiation successful
0809 05:38:20.677226 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":256,"threads_started":1}
I20260809 05:38:20.678799 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:20.658283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:54041)
0809 05:38:20.659166 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:20.659213 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:20.661120 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:20.661852 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:20.661860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:20.662377 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:20.663018 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:20.663041 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:20.664573 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:20.664577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:20.665826 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:20.665838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:20.666922 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:20.666927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:20.667124 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:20.667875 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:20.667899 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:20.670459 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:20.674548 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:20.674555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:20.674568 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:20.675062 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:20.675477 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:20.675481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:20.675487 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:20.675678 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:20.676250 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:20.676257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:20.676602 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 05:38:20.676869 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 05:38:20.677420 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":401,"threads_started":1}
I20260809 05:38:20.680454 19986 heartbeater.cc:347] Connected to a master server at 127.30.137.254:38275
I20260809 05:38:20.680820 19986 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:20.681895 19986 heartbeater.cc:511] Master 127.30.137.254:38275 requested a full tablet report, sending...
I20260809 05:38:20.684559 19708 ts_manager.cc:194] Registered new tserver with Master: 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193:44221)
I20260809 05:38:20.686932 19708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:54041
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:38:20.730021 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:38275
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:21.088603 19995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:21.088905 19995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:21.089007 19995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:21.096998 19995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:21.097143 19995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:21.097261 19995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:38:21.107489 19995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:21.108825 19995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:21.110293 19995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:21.124457 20007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:21.124504 20003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:21.124411 20001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:21.131054 19995 server_base.cc:1034] running on GCE node
I20260809 05:38:21.131836 19995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:21.132858 19995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:21.134074 19995 hybrid_clock.cc:648] HybridClock initialized: now 1786253901134025 us; error 44 us; skew 500 ppm
I20260809 05:38:21.134430 19995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:21 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:21.137953 19995 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:38:21.139698 19995 webserver.cc:460] Webserver started at http://127.30.137.194:37359/ using document root <none> and password file <none>
I20260809 05:38:21.140499 19995 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:21.140671 19995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:21.141014 19995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:21.144227 19995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "dd22a230f5ee474583c95dc778c59036"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "07293913c20187f2aed42c0e93656a96"
server_key_iv: "64b157d4eb0dc9267b95b112be83c471"
server_key_version: "encryptionkey@0"
I20260809 05:38:21.145087 19995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "dd22a230f5ee474583c95dc778c59036"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "07293913c20187f2aed42c0e93656a96"
server_key_iv: "64b157d4eb0dc9267b95b112be83c471"
server_key_version: "encryptionkey@0"
I20260809 05:38:21.150964 19995 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 05:38:21.154748 20016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:21.156263 19995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:38:21.156452 19995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "dd22a230f5ee474583c95dc778c59036"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "07293913c20187f2aed42c0e93656a96"
server_key_iv: "64b157d4eb0dc9267b95b112be83c471"
server_key_version: "encryptionkey@0"
I20260809 05:38:21.156677 19995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:21.193715 19995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:21.198081 19995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:21.198415 19995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:21.199555 19995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:21.201081 19995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:21.201156 19995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:21.201341 19995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:21.201397 19995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:21.237497 19995 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:40229
I20260809 05:38:21.237551 20174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:40229 every 8 connection(s)
I20260809 05:38:21.238924 19995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:38:21.241814 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 19995
I20260809 05:38:21.242035 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:38:21.242739 31271 external_mini_cluster.cc:1428] Setting key 2d031339e82badd884fe0624b94f40bc
Aug 09 05:38:21 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:21.262213 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.242186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:54991 (local address 127.30.137.254:38275)
0809 05:38:21.242718 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:21.242725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:21.243981 (+ 1256us) server_negotiation.cc:407] Connection header received
0809 05:38:21.246042 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:21.246048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:21.246144 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:21.246274 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:21.248056 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.248849 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.250441 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.250963 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.251096 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.254969 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:21.254978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:21.254986 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:21.255036 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:21.258523 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:21.259543 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:21.259547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:21.259550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:21.259678 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:21.260212 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:21.260219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:21.260222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:21.260605 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:21.260723 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:21.261743 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0809 05:38:21.261935 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":228,"threads_started":1}
I20260809 05:38:21.263870 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.242920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:54991)
0809 05:38:21.243806 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:21.243887 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:21.245744 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:21.246536 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:21.246544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:21.247248 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:21.247879 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.247899 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.249019 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.249024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.250283 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.250296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.251211 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.251219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.251454 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.252220 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:21.252247 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:21.254714 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:21.258695 (+ 3981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:21.258708 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:21.258731 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:21.259410 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:21.259794 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:21.259801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:21.259808 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:21.260074 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:21.260742 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:21.260753 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:21.261224 (+ 471us) client_negotiation.cc:769] Sending connection context
0809 05:38:21.261607 (+ 383us) client_negotiation.cc:240] Negotiation successful
0809 05:38:21.262260 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":383,"threads_started":1}
I20260809 05:38:21.265404 20175 heartbeater.cc:347] Connected to a master server at 127.30.137.254:38275
I20260809 05:38:21.265766 20175 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:21.266736 20175 heartbeater.cc:511] Master 127.30.137.254:38275 requested a full tablet report, sending...
I20260809 05:38:21.268393 19708 ts_manager.cc:194] Registered new tserver with Master: dd22a230f5ee474583c95dc778c59036 (127.30.137.194:40229)
I20260809 05:38:21.269245 19708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:54991
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:38:21.313951 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:38275
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:21.690994 19986 heartbeater.cc:503] Master 127.30.137.254:38275 was elected leader, sending a full tablet report...
W20260809 05:38:21.695320 20187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:21.695619 20187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:21.695815 20187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:21.703351 20187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:21.703480 20187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:21.703698 20187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:38:21.714031 20187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:21.715624 20187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:21.717082 20187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:21.731498 20198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:21.731429 20200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:21.731511 20197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:21.732781 20187 server_base.cc:1034] running on GCE node
I20260809 05:38:21.733443 20187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:21.734460 20187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:21.735708 20187 hybrid_clock.cc:648] HybridClock initialized: now 1786253901735695 us; error 64 us; skew 500 ppm
I20260809 05:38:21.736070 20187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:21 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:21.739533 20187 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:38:21.741314 20187 webserver.cc:460] Webserver started at http://127.30.137.195:45717/ using document root <none> and password file <none>
I20260809 05:38:21.742051 20187 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:21.742210 20187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:21.742619 20187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:21.745883 20187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "08c8dba24bec4339b99057da06062fc6"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "ec9fe12faa149f289a787908d55cfcb5"
server_key_iv: "39b49561381ea06485f15c0113a3f847"
server_key_version: "encryptionkey@0"
I20260809 05:38:21.746862 20187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "08c8dba24bec4339b99057da06062fc6"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "ec9fe12faa149f289a787908d55cfcb5"
server_key_iv: "39b49561381ea06485f15c0113a3f847"
server_key_version: "encryptionkey@0"
I20260809 05:38:21.752300 20187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:38:21.756001 20211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:21.757457 20187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:38:21.757743 20187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "08c8dba24bec4339b99057da06062fc6"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "ec9fe12faa149f289a787908d55cfcb5"
server_key_iv: "39b49561381ea06485f15c0113a3f847"
server_key_version: "encryptionkey@0"
I20260809 05:38:21.757977 20187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:21.781788 20187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:21.786624 20187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:21.787045 20187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:21.788105 20187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:21.789583 20187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:21.789664 20187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:21.789844 20187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:21.789913 20187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:21.825477 20187 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40305
I20260809 05:38:21.825531 20367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40305 every 8 connection(s)
I20260809 05:38:21.827091 20187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:38:21.836788 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 20187
I20260809 05:38:21.837028 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:38:21.837519 31271 external_mini_cluster.cc:1428] Setting key c6b5cb05803eb502b0525322ff76d69f
Aug 09 05:38:21 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:21.848883 20374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.829892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:32829 (local address 127.30.137.254:38275)
0809 05:38:21.830427 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:21.830431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:21.831337 (+ 906us) server_negotiation.cc:407] Connection header received
0809 05:38:21.833311 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:21.833319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:21.833431 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:21.833562 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:21.835506 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.836347 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.838169 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.838640 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.838795 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.843163 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:21.843168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:21.843172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:21.843208 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:21.845774 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:21.846570 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:21.846575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:21.846577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:21.846752 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:21.847178 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:21.847181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:21.847183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:21.847521 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:21.847640 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:21.848389 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 05:38:21.848607 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":283,"threads_started":1}
I20260809 05:38:21.849856 20375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.830373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:32829)
0809 05:38:21.831164 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:21.831210 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:21.833020 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:21.833822 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:21.833830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:21.834592 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:21.835310 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.835330 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.836539 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.836544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.838003 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.838017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.838918 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.838927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.839260 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.840135 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:21.840157 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:21.842959 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:21.845912 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:21.845919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:21.845931 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:21.846455 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:21.846858 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:21.846861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:21.846863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:21.847071 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:21.847635 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:21.847641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:21.848032 (+ 391us) client_negotiation.cc:769] Sending connection context
0809 05:38:21.848274 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 05:38:21.848682 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":336,"threads_started":1}
I20260809 05:38:21.851384 20370 heartbeater.cc:347] Connected to a master server at 127.30.137.254:38275
I20260809 05:38:21.851768 20370 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:21.852731 20370 heartbeater.cc:511] Master 127.30.137.254:38275 requested a full tablet report, sending...
I20260809 05:38:21.854328 19708 ts_manager.cc:194] Registered new tserver with Master: 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305)
I20260809 05:38:21.855161 19708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:32829
I20260809 05:38:21.866153 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:38:21.884219 20374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.871056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.30.137.254:38275)
0809 05:38:21.871302 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:21.871310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:21.871704 (+ 394us) server_negotiation.cc:407] Connection header received
0809 05:38:21.871825 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:21.871829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:21.871917 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:21.872019 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:21.873622 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.874443 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.875811 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.876229 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.876335 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.878850 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:21.878856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:21.878859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:21.878896 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:21.881649 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:21.882320 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:21.882324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:21.882326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:21.882448 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:21.882803 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:21.882808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:21.882810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:21.883175 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:21.883382 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:21.883672 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 05:38:21.883864 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 05:38:21.888545 19708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:38:21.896029 19708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:38:21.935603 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.910135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40305 (local address 127.0.0.1:53910)
0809 05:38:21.926507 (+ 16372us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:21.926526 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:21.926800 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:21.927601 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:21.927609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:21.927639 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:21.928216 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.928232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.930498 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.930506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.932961 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.932977 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.934382 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.934388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.934583 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.934701 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 05:38:21.934886 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 05:38:21.935091 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16236,"mutex_wait_us":1,"thread_start_us":17970,"threads_started":1}
I20260809 05:38:21.936673 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.910887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53910 (local address 127.30.137.195:40305)
0809 05:38:21.918355 (+ 7468us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:21.918369 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:21.926681 (+ 8312us) server_negotiation.cc:407] Connection header received
0809 05:38:21.927027 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:21.927038 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:21.927392 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:21.927594 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:21.928575 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.930318 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.933143 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.934137 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.934384 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.934575 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:21.934935 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 05:38:21.935112 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7205,"thread_start_us":6982,"threads_started":1}
I20260809 05:38:21.938139 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.910446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:40229 (local address 127.0.0.1:52676)
0809 05:38:21.928470 (+ 18024us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:21.928490 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:21.928608 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:21.929417 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:21.929422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:21.929448 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:21.930425 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.930445 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.932449 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.932456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.935171 (+ 2715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.935188 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.937157 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.937163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.937348 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.937406 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 05:38:21.937513 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 05:38:21.937655 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17886,"mutex_wait_us":9,"thread_start_us":13442,"threads_started":1}
I20260809 05:38:21.938231 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.909726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41752)
0809 05:38:21.924463 (+ 14737us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:21.924544 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:21.924864 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:21.929742 (+ 4878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:21.929749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:21.929779 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:21.930797 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.930812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.932238 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.932246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.935344 (+ 3098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:21.935356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.937421 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.937427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:21.937605 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.937635 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 05:38:21.937703 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 05:38:21.937840 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14345,"mutex_wait_us":544,"thread_start_us":16471,"threads_started":1}
I20260809 05:38:21.938587 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.910061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.30.137.193:44221)
0809 05:38:21.929164 (+ 19103us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:21.929173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:21.929204 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:38:21.929310 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:21.929317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:21.929536 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:21.929776 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:21.931050 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.932083 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.935539 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.936822 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.937074 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.937265 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:21.938066 (+ 801us) server_negotiation.cc:299] Negotiation successful
0809 05:38:21.938242 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18915,"thread_start_us":18738,"threads_started":1}
I20260809 05:38:21.939072 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:21.911110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.30.137.194:40229)
0809 05:38:21.919449 (+ 8339us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:21.919457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:21.928598 (+ 9141us) server_negotiation.cc:407] Connection header received
0809 05:38:21.928815 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:21.928825 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:21.929106 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:21.929288 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:21.930689 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.932337 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.935332 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:21.936924 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:21.937111 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:21.937278 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:21.937574 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 05:38:21.937751 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8163,"thread_start_us":8034,"threads_started":1}
I20260809 05:38:21.941210 20281 tablet_service.cc:1467] Processing CreateTablet for tablet 01d8e702b26d47d88e232fc11de9a792 (DEFAULT_TABLE table=test-table [id=7c673fd5ebea40e18c2689d992df5998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:21.942396 20089 tablet_service.cc:1467] Processing CreateTablet for tablet 01d8e702b26d47d88e232fc11de9a792 (DEFAULT_TABLE table=test-table [id=7c673fd5ebea40e18c2689d992df5998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:21.942871 20281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d8e702b26d47d88e232fc11de9a792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:21.942433 19894 tablet_service.cc:1467] Processing CreateTablet for tablet 01d8e702b26d47d88e232fc11de9a792 (DEFAULT_TABLE table=test-table [id=7c673fd5ebea40e18c2689d992df5998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:21.943548 20089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d8e702b26d47d88e232fc11de9a792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:21.943858 19894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d8e702b26d47d88e232fc11de9a792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:21.956004 20403 tablet_bootstrap.cc:492] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Bootstrap starting.
I20260809 05:38:21.958791 20406 tablet_bootstrap.cc:492] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Bootstrap starting.
I20260809 05:38:21.959897 20407 tablet_bootstrap.cc:492] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Bootstrap starting.
I20260809 05:38:21.962231 20403 tablet_bootstrap.cc:654] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:21.963582 20407 tablet_bootstrap.cc:654] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:21.963594 20406 tablet_bootstrap.cc:654] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:21.963726 20403 log.cc:824] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:21.964905 20407 log.cc:824] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:21.964895 20406 log.cc:824] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:21.967095 20403 tablet_bootstrap.cc:492] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: No bootstrap required, opened a new log
I20260809 05:38:21.967398 20403 ts_tablet_manager.cc:1397] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 05:38:21.968060 20407 tablet_bootstrap.cc:492] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: No bootstrap required, opened a new log
I20260809 05:38:21.968395 20406 tablet_bootstrap.cc:492] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: No bootstrap required, opened a new log
I20260809 05:38:21.968428 20407 ts_tablet_manager.cc:1397] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260809 05:38:21.968693 20406 ts_tablet_manager.cc:1397] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260809 05:38:21.976323 20407 raft_consensus.cc:348] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:21.976765 20407 raft_consensus.cc:374] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:21.976941 20407 raft_consensus.cc:729] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c8dba24bec4339b99057da06062fc6, State: Initialized, Role: FOLLOWER
I20260809 05:38:21.977596 20407 consensus_queue.cc:260] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:21.978446 20403 raft_consensus.cc:348] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:21.978860 20370 heartbeater.cc:503] Master 127.30.137.254:38275 was elected leader, sending a full tablet report...
I20260809 05:38:21.979064 20403 raft_consensus.cc:374] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:21.979220 20403 raft_consensus.cc:729] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd22a230f5ee474583c95dc778c59036, State: Initialized, Role: FOLLOWER
I20260809 05:38:21.979604 20407 ts_tablet_manager.cc:1428] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260809 05:38:21.979633 20406 raft_consensus.cc:348] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:21.980087 20406 raft_consensus.cc:374] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:21.979991 20403 consensus_queue.cc:260] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:21.980223 20406 raft_consensus.cc:729] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12342fed55b44c7883d4b64bb5f554d7, State: Initialized, Role: FOLLOWER
I20260809 05:38:21.980914 20406 consensus_queue.cc:260] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:21.982990 20175 heartbeater.cc:503] Master 127.30.137.254:38275 was elected leader, sending a full tablet report...
I20260809 05:38:21.984004 20403 ts_tablet_manager.cc:1428] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260809 05:38:21.985831 20406 ts_tablet_manager.cc:1428] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
W20260809 05:38:21.991067 20178 tablet.cc:2369] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:22.078974 20372 tablet.cc:2369] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:22.157980 19987 tablet.cc:2369] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:38:22.375979 20414 raft_consensus.cc:482] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:38:22.376351 20414 raft_consensus.cc:504] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:22.378141 20414 leader_election.cc:288] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193:44221), 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305)
I20260809 05:38:22.385608 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.378699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:44191 (local address 127.30.137.193:44221)
0809 05:38:22.378950 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:22.378958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:22.379729 (+ 771us) server_negotiation.cc:407] Connection header received
0809 05:38:22.379956 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:22.379962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:22.380048 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:22.380133 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:22.380871 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.381626 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.383685 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.384539 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.384722 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.384818 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:22.385185 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 05:38:22.385350 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 05:38:22.385768 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.378606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:50147 (local address 127.30.137.195:40305)
0809 05:38:22.378851 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:22.378860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:22.379807 (+ 947us) server_negotiation.cc:407] Connection header received
0809 05:38:22.380031 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:22.380035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:22.380125 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:22.380230 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:22.380943 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.381651 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.383701 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.384797 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.384961 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.385030 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:22.385379 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 05:38:22.385512 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 05:38:22.386000 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.378591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.30.137.194:44191)
0809 05:38:22.379579 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:22.379600 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:22.379776 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:22.380186 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:22.380192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:22.380228 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:22.380699 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:22.380713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.381775 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.381782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:22.383519 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:22.383532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.384772 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.384779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:22.384990 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.385026 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 05:38:22.385125 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 05:38:22.385227 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":87,"thread_start_us":515,"threads_started":1}
I20260809 05:38:22.386098 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.378740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40305 (local address 127.30.137.194:50147)
0809 05:38:22.379648 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:22.379687 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:22.379868 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:22.380272 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:22.380278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:22.380310 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:22.380727 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:22.380737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.381824 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.381827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:22.383517 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:22.383530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.385024 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.385028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:22.385186 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.385218 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:38:22.385310 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 05:38:22.385428 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":7,"thread_start_us":538,"threads_started":1}
I20260809 05:38:22.387370 20307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d8e702b26d47d88e232fc11de9a792" candidate_uuid: "dd22a230f5ee474583c95dc778c59036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c8dba24bec4339b99057da06062fc6" is_pre_election: true
I20260809 05:38:22.387392 19927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d8e702b26d47d88e232fc11de9a792" candidate_uuid: "dd22a230f5ee474583c95dc778c59036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12342fed55b44c7883d4b64bb5f554d7" is_pre_election: true
I20260809 05:38:22.387873 20307 raft_consensus.cc:2454] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd22a230f5ee474583c95dc778c59036 in term 0.
I20260809 05:38:22.387878 19927 raft_consensus.cc:2454] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd22a230f5ee474583c95dc778c59036 in term 0.
I20260809 05:38:22.388538 20018 leader_election.cc:302] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c8dba24bec4339b99057da06062fc6, dd22a230f5ee474583c95dc778c59036; no voters:
I20260809 05:38:22.388975 20414 raft_consensus.cc:2781] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:38:22.389153 20414 raft_consensus.cc:482] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:38:22.389267 20414 raft_consensus.cc:3037] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:22.391719 20414 raft_consensus.cc:504] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:22.392419 20414 leader_election.cc:288] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [CANDIDATE]: Term 1 election: Requested vote from peers 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193:44221), 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305)
I20260809 05:38:22.393025 19927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d8e702b26d47d88e232fc11de9a792" candidate_uuid: "dd22a230f5ee474583c95dc778c59036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12342fed55b44c7883d4b64bb5f554d7"
I20260809 05:38:22.393059 20307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d8e702b26d47d88e232fc11de9a792" candidate_uuid: "dd22a230f5ee474583c95dc778c59036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c8dba24bec4339b99057da06062fc6"
I20260809 05:38:22.393226 19927 raft_consensus.cc:3037] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:22.393369 20307 raft_consensus.cc:3037] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:22.395614 19927 raft_consensus.cc:2454] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd22a230f5ee474583c95dc778c59036 in term 1.
I20260809 05:38:22.396090 20307 raft_consensus.cc:2454] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd22a230f5ee474583c95dc778c59036 in term 1.
I20260809 05:38:22.396049 20021 leader_election.cc:302] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12342fed55b44c7883d4b64bb5f554d7, dd22a230f5ee474583c95dc778c59036; no voters:
I20260809 05:38:22.396452 20414 raft_consensus.cc:2781] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:22.396974 20414 raft_consensus.cc:686] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 1 LEADER]: Becoming Leader. State: Replica: dd22a230f5ee474583c95dc778c59036, State: Running, Role: LEADER
I20260809 05:38:22.397475 20414 consensus_queue.cc:237] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } }
I20260809 05:38:22.402323 19708 catalog_manager.cc:5495] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd22a230f5ee474583c95dc778c59036 (127.30.137.194). New cstate: current_term: 1 leader_uuid: "dd22a230f5ee474583c95dc778c59036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } health_report { overall_health: HEALTHY } } }
I20260809 05:38:22.509979 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.504258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.30.137.194:40229)
0809 05:38:22.504749 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:22.504754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:22.504842 (+ 88us) server_negotiation.cc:407] Connection header received
0809 05:38:22.505036 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:22.505040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:22.505134 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:22.505294 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:38:22.505685 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.506377 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.507904 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.508323 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.508427 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.508927 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:38:22.509464 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:38:22.509581 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:22.509684 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 05:38:22.509756 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
I20260809 05:38:22.518347 20307 raft_consensus.cc:1261] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Refusing update from remote peer dd22a230f5ee474583c95dc778c59036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:22.518347 19927 raft_consensus.cc:1261] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Refusing update from remote peer dd22a230f5ee474583c95dc778c59036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:22.519734 20414 consensus_queue.cc:1035] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:22.520105 20419 consensus_queue.cc:1035] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:22.558596 19708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:22.558975 19708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:22.563354 19708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Sending DeleteTablet for 3 replicas of tablet 01d8e702b26d47d88e232fc11de9a792
I20260809 05:38:22.564739 19894 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d8e702b26d47d88e232fc11de9a792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:22 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:41752
I20260809 05:38:22.564739 20281 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d8e702b26d47d88e232fc11de9a792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:22 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:53910
I20260809 05:38:22.565606 20089 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d8e702b26d47d88e232fc11de9a792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:22 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:52676
I20260809 05:38:22.565744 20442 tablet_replica.cc:330] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: stopping tablet replica
I20260809 05:38:22.566216 20442 raft_consensus.cc:2229] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:22.566828 20442 raft_consensus.cc:2258] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:22.568620 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19798
I20260809 05:38:22.569193 20442 ts_tablet_manager.cc:1905] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:22.571208 20443 tablet_replica.cc:330] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: stopping tablet replica
I20260809 05:38:22.571723 20443 raft_consensus.cc:2229] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:22.572284 20443 raft_consensus.cc:2258] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:22.579125 20442 ts_tablet_manager.cc:1918] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:22.580338 20444 tablet_replica.cc:330] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: stopping tablet replica
I20260809 05:38:22.580998 20443 ts_tablet_manager.cc:1905] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:22.580981 20444 raft_consensus.cc:2229] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:38:22.581769 20444 raft_consensus.cc:2258] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:22.584097 20444 ts_tablet_manager.cc:1905] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:22.588248 20442 log.cc:1196] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/01d8e702b26d47d88e232fc11de9a792
I20260809 05:38:22.588711 20443 ts_tablet_manager.cc:1918] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:22.588971 20442 ts_tablet_manager.cc:1939] T 01d8e702b26d47d88e232fc11de9a792 P 12342fed55b44c7883d4b64bb5f554d7: Deleting consensus metadata
I20260809 05:38:22.589020 20443 log.cc:1196] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/01d8e702b26d47d88e232fc11de9a792
I20260809 05:38:22.589708 20443 ts_tablet_manager.cc:1939] T 01d8e702b26d47d88e232fc11de9a792 P 08c8dba24bec4339b99057da06062fc6: Deleting consensus metadata
I20260809 05:38:22.591271 19688 catalog_manager.cc:4841] TS 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193:44221): tablet 01d8e702b26d47d88e232fc11de9a792 (table test-table [id=7c673fd5ebea40e18c2689d992df5998]) successfully deleted
I20260809 05:38:22.591713 19687 catalog_manager.cc:4841] TS 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305): tablet 01d8e702b26d47d88e232fc11de9a792 (table test-table [id=7c673fd5ebea40e18c2689d992df5998]) successfully deleted
I20260809 05:38:22.592640 20444 ts_tablet_manager.cc:1918] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:22.592892 20444 log.cc:1196] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/01d8e702b26d47d88e232fc11de9a792
I20260809 05:38:22.593421 20444 ts_tablet_manager.cc:1939] T 01d8e702b26d47d88e232fc11de9a792 P dd22a230f5ee474583c95dc778c59036: Deleting consensus metadata
I20260809 05:38:22.595165 19686 catalog_manager.cc:4841] TS dd22a230f5ee474583c95dc778c59036 (127.30.137.194:40229): tablet 01d8e702b26d47d88e232fc11de9a792 (table test-table [id=7c673fd5ebea40e18c2689d992df5998]) successfully deleted
Aug 09 05:38:22 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:38:22.604426 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.570510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.30.137.193:44221)
0809 05:38:22.570770 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:22.570779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:22.579486 (+ 8707us) server_negotiation.cc:407] Connection header received
0809 05:38:22.579749 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:22.579755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:22.579855 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:22.579974 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:22.582587 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.583835 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.585864 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.586522 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.586638 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.597906 (+ 11268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:22.597923 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:22.597939 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:22.597981 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:22.601166 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:22.602132 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:22.602140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:22.602142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:22.602293 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:22.602823 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:22.602827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:22.602829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:22.603220 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:22.603474 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:22.603926 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 05:38:22.604113 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 05:38:22.605430 19980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:22.699172 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19798
I20260809 05:38:22.725147 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19995
Aug 09 05:38:22 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:38:22.748250 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.726069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.30.137.194:40229)
0809 05:38:22.726257 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:22.726267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:22.726312 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:38:22.726565 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:22.726571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:22.726711 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:22.726832 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:22.728263 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.729100 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.730507 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.731165 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.731331 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.740867 (+ 9536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:22.740883 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:22.740910 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:22.740959 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:22.744782 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:22.745694 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:22.745703 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:22.745705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:22.745860 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:22.746368 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:22.746374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:22.746377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:22.746854 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:22.747214 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:22.747611 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 05:38:22.747877 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:38:22.749102 20169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:22.870033 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19995
I20260809 05:38:22.896382 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 20187
Aug 09 05:38:22 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:38:22.912385 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:22.897174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.30.137.195:40305)
0809 05:38:22.897639 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:22.897643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:22.897661 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:38:22.897719 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:22.897723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:22.897790 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:22.897883 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:22.899222 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.900019 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.901290 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:22.901706 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:22.901802 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:22.906398 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:22.906417 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:22.906433 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:22.906470 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:22.909647 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:22.910365 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:22.910373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:22.910375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:22.910508 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:22.910926 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:22.910929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:22.910931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:22.911323 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:22.911575 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:22.911801 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 05:38:22.911995 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":256,"threads_started":1}
I20260809 05:38:22.913152 20361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:23.010913 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 20187
I20260809 05:38:23.034162 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 19667
I20260809 05:38:23.035439 19749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:23.149691 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 19667
I20260809 05:38:23.179862 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:38275
--webserver_interface=127.30.137.254
--webserver_port=43599
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:38275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:23.551632 20452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:23.552058 20452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:23.552234 20452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:23.559926 20452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:23.560102 20452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:23.560213 20452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:23.560333 20452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:23.560422 20452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:23.571120 20452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:38275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:38275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=43599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:23.572559 20452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:23.573900 20452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:23.607064 20459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:23.607388 20458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:23.607800 20461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:23.609088 20452 server_base.cc:1034] running on GCE node
I20260809 05:38:23.610052 20452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:23.611837 20452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:23.613059 20452 hybrid_clock.cc:648] HybridClock initialized: now 1786253903613009 us; error 45 us; skew 500 ppm
I20260809 05:38:23.613509 20452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:23 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:23.617317 20452 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:23.619161 20452 webserver.cc:460] Webserver started at http://127.30.137.254:43599/ using document root <none> and password file <none>
I20260809 05:38:23.620023 20452 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:23.620151 20452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:23.626328 20452 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 05:38:23.629465 20472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:23.630945 20452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:38:23.631145 20452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "d111d9a125d249f9b712fe1819fb1f38"
format_stamp: "Formatted at 2026-08-09 05:38:19 on dist-test-slave-3kk6"
server_key: "d41f6b7d6d7d147f5d70d13e98f1d3bf"
server_key_iv: "1109dcd15c0a135ca653c47cea921f1a"
server_key_version: "encryptionkey@0"
I20260809 05:38:23.632289 20452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:23.650835 20452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:23.655817 20452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:23.656147 20452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:23.676959 20452 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:38275
I20260809 05:38:23.676990 20548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:38275 every 8 connection(s)
I20260809 05:38:23.678750 20452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:23.680991 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 20452
I20260809 05:38:23.681895 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:44221
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:38275
--webserver_port=45005
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:23.686401 20549 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:38:23.689939 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap starting.
I20260809 05:38:23.710453 20549 log.cc:824] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:23.723246 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:38:23.723851 20549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap complete.
I20260809 05:38:23.733752 20549 raft_consensus.cc:348] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:23.734758 20549 raft_consensus.cc:729] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d111d9a125d249f9b712fe1819fb1f38, State: Initialized, Role: FOLLOWER
I20260809 05:38:23.735379 20549 consensus_queue.cc:260] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:23.735654 20549 raft_consensus.cc:388] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:23.735841 20549 raft_consensus.cc:482] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:23.736052 20549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:38:23.740407 20549 raft_consensus.cc:504] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:23.741014 20549 leader_election.cc:302] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d111d9a125d249f9b712fe1819fb1f38; no voters:
I20260809 05:38:23.741710 20549 leader_election.cc:288] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 05:38:23.741870 20554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 05:38:23.743147 20554 raft_consensus.cc:686] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 LEADER]: Becoming Leader. State: Replica: d111d9a125d249f9b712fe1819fb1f38, State: Running, Role: LEADER
I20260809 05:38:23.743831 20554 consensus_queue.cc:237] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:23.744249 20549 sys_catalog.cc:564] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:23.746320 20557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d111d9a125d249f9b712fe1819fb1f38. Latest consensus state: current_term: 2 leader_uuid: "d111d9a125d249f9b712fe1819fb1f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } } }
I20260809 05:38:23.746747 20557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:23.747340 20555 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d111d9a125d249f9b712fe1819fb1f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } } }
I20260809 05:38:23.747648 20555 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:23.752728 20563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:23.762486 20563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:23.764109 20563 catalog_manager.cc:1259] Loaded cluster ID: 9fce34ef1a524b16bd7d1715151b8761
I20260809 05:38:23.764227 20563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:23.767691 20563 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 05:38:23.786618 20563 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 05:38:23.796643 20563 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:23.798497 20563 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Loaded TSK: 0
I20260809 05:38:23.800100 20563 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 05:38:24.054159 20551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:24.054580 20551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:24.054804 20551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:24.062379 20551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:24.062546 20551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:24.062780 20551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:38:24.073372 20551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:44221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=45005
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:24.074901 20551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:24.076308 20551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:24.093164 20591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:24.094322 20590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:24.094515 20594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:24.095046 20551 server_base.cc:1034] running on GCE node
I20260809 05:38:24.095991 20551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:24.098996 20551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:24.100381 20551 hybrid_clock.cc:648] HybridClock initialized: now 1786253904100291 us; error 52 us; skew 500 ppm
I20260809 05:38:24.100826 20551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:24 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:24.106870 20551 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:38:24.108912 20551 webserver.cc:460] Webserver started at http://127.30.137.193:45005/ using document root <none> and password file <none>
I20260809 05:38:24.109901 20551 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:24.110085 20551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:24.117964 20551 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:38:24.121428 20602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:24.123308 20551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:24.123520 20551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "12342fed55b44c7883d4b64bb5f554d7"
format_stamp: "Formatted at 2026-08-09 05:38:20 on dist-test-slave-3kk6"
server_key: "72b5098cb84ddc3f3734d27da9c71402"
server_key_iv: "afe68270e468f5944c1849d1aa8e29fa"
server_key_version: "encryptionkey@0"
I20260809 05:38:24.124948 20551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:24.153129 20551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:24.157716 20551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:24.157975 20551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:24.159076 20551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:24.160575 20551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:24.160709 20551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:24.160847 20551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:24.160948 20551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:24.198369 20551 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44221
I20260809 05:38:24.198469 20754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44221 every 8 connection(s)
I20260809 05:38:24.199983 20551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:38:24.203722 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 20551
I20260809 05:38:24.204459 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:40229
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:38275
--webserver_port=37359
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:38:24 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:24.224706 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:24.203523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:54547)
0809 05:38:24.204502 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:24.204558 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:24.206377 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:24.207869 (+ 1492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:24.207880 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:24.208522 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:24.209086 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:24.209105 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.210841 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.210845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:24.212111 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:24.212124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.213240 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.213248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:24.213523 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:24.214265 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:24.214296 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:24.216586 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:24.220149 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:24.220159 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:24.220174 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:24.220614 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:24.221087 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:24.221092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:24.221096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:24.221285 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:24.222069 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:24.222077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:24.222419 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 05:38:24.222730 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 05:38:24.223321 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":465,"threads_started":1}
I20260809 05:38:24.224720 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:24.203171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:54547 (local address 127.30.137.254:38275)
0809 05:38:24.203910 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:24.203928 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:24.204950 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 05:38:24.206965 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:24.206978 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:24.207323 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:24.207696 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:24.209348 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.210607 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.212268 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.212961 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.213189 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:24.216882 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:24.216892 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:24.216907 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:24.216946 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:24.219993 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:24.220759 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:24.220775 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:24.220782 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:24.220949 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:24.221402 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:24.221407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:24.221411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:24.221911 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:24.222132 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:24.222932 (+ 800us) server_negotiation.cc:299] Negotiation successful
0809 05:38:24.223347 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":299,"threads_started":1}
I20260809 05:38:24.227803 20757 heartbeater.cc:347] Connected to a master server at 127.30.137.254:38275
I20260809 05:38:24.228178 20757 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:24.228956 20757 heartbeater.cc:511] Master 127.30.137.254:38275 requested a full tablet report, sending...
I20260809 05:38:24.231393 20496 ts_manager.cc:194] Registered new tserver with Master: 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193:44221)
I20260809 05:38:24.232913 20496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:54547
W20260809 05:38:24.564514 20763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:24.564908 20763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:24.565088 20763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:24.573026 20763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:24.573220 20763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:24.573421 20763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:38:24.583976 20763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:40229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=37359
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:24.585507 20763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:24.587249 20763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:24.600875 20772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:24.602202 20773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:24.602339 20776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:24.603315 20763 server_base.cc:1034] running on GCE node
I20260809 05:38:24.603991 20763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:24.605077 20763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:24.606330 20763 hybrid_clock.cc:648] HybridClock initialized: now 1786253904606267 us; error 67 us; skew 500 ppm
I20260809 05:38:24.606761 20763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:24 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:24.610174 20763 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:38:24.611801 20763 webserver.cc:460] Webserver started at http://127.30.137.194:37359/ using document root <none> and password file <none>
I20260809 05:38:24.612558 20763 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:24.612735 20763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:24.618577 20763 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:38:24.621593 20789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:24.623112 20763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:24.623317 20763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "dd22a230f5ee474583c95dc778c59036"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "07293913c20187f2aed42c0e93656a96"
server_key_iv: "64b157d4eb0dc9267b95b112be83c471"
server_key_version: "encryptionkey@0"
I20260809 05:38:24.624418 20763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:24.639098 20763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:24.644032 20763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:24.644420 20763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:24.645568 20763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:24.647181 20763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:24.647318 20763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:24.647456 20763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:24.647557 20763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:24.684134 20763 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:40229
I20260809 05:38:24.684185 20950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:40229 every 8 connection(s)
I20260809 05:38:24.685679 20763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:38:24.687206 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 20763
I20260809 05:38:24.687914 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:40305
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:38275
--webserver_port=45717
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:38:24 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:24.708292 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:24.688680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:43217 (local address 127.30.137.254:38275)
0809 05:38:24.688896 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:24.688902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:24.690351 (+ 1449us) server_negotiation.cc:407] Connection header received
0809 05:38:24.692607 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:24.692614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:24.692737 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:24.692858 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:24.694485 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.695295 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.696949 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.697434 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.697580 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:24.701692 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:24.701700 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:24.701707 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:24.701757 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:24.704827 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:24.705656 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:24.705662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:24.705666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:24.705826 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:24.706310 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:24.706316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:24.706320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:24.706625 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:24.706872 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:24.707644 (+ 772us) server_negotiation.cc:299] Negotiation successful
0809 05:38:24.707916 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 05:38:24.709421 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:24.689294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:43217)
0809 05:38:24.690164 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:24.690212 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:24.692284 (+ 2072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:24.693134 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:24.693142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:24.693696 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:24.694289 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:24.694307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.695475 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.695480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:24.696770 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:24.696783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:24.697691 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:24.697699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:24.697961 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:24.698805 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:24.698858 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:24.701438 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:24.705009 (+ 3571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:24.705019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:24.705038 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:24.705519 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:24.705956 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:24.705961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:24.705965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:24.706184 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:24.706832 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:24.706841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:24.707230 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 05:38:24.707536 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 05:38:24.708133 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"spinlock_wait_cycles":15744,"thread_start_us":444,"threads_started":1}
I20260809 05:38:24.711118 20951 heartbeater.cc:347] Connected to a master server at 127.30.137.254:38275
I20260809 05:38:24.711458 20951 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:24.712262 20951 heartbeater.cc:511] Master 127.30.137.254:38275 requested a full tablet report, sending...
I20260809 05:38:24.713927 20496 ts_manager.cc:194] Registered new tserver with Master: dd22a230f5ee474583c95dc778c59036 (127.30.137.194:40229)
I20260809 05:38:24.715368 20496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:43217
W20260809 05:38:25.052982 20958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:25.053341 20958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:25.053515 20958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:25.061071 20958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:25.061201 20958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:25.061327 20958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:38:25.071564 20958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:40305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=45717
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:25.072917 20958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:25.074211 20958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:25.088133 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:25.088099 20968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:25.088210 20964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:25.089386 20958 server_base.cc:1034] running on GCE node
I20260809 05:38:25.090046 20958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:25.091114 20958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:25.092329 20958 hybrid_clock.cc:648] HybridClock initialized: now 1786253905092258 us; error 61 us; skew 500 ppm
I20260809 05:38:25.092659 20958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:25 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:25.096195 20958 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:38:25.097776 20958 webserver.cc:460] Webserver started at http://127.30.137.195:45717/ using document root <none> and password file <none>
I20260809 05:38:25.098496 20958 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:25.098726 20958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:25.104254 20958 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 05:38:25.107057 20979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:25.108420 20958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:38:25.108613 20958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "08c8dba24bec4339b99057da06062fc6"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "ec9fe12faa149f289a787908d55cfcb5"
server_key_iv: "39b49561381ea06485f15c0113a3f847"
server_key_version: "encryptionkey@0"
I20260809 05:38:25.109663 20958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:25.139307 20958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:25.144227 20958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:25.144590 20958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:25.145713 20958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:25.147315 20958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:25.147416 20958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:25.147501 20958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:25.147559 20958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:25.183177 20958 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40305
I20260809 05:38:25.183322 21134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40305 every 8 connection(s)
I20260809 05:38:25.184759 20958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:38:25.193778 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 20958
Aug 09 05:38:25 dist-test-slave-3kk6 krb5kdc[19646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:25.209096 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.187515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:37235 (local address 127.30.137.254:38275)
0809 05:38:25.187712 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.187717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.188895 (+ 1178us) server_negotiation.cc:407] Connection header received
0809 05:38:25.190884 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.190890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.190992 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.191110 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:25.192803 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.193558 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.195699 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.196434 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.196586 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.201432 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:25.201443 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:25.201450 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:25.201493 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:25.205158 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.206237 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.206241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.206244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.206410 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.207035 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.207039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.207041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.207351 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:25.207466 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.208476 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.208706 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:38:25.210103 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.187992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:37235)
0809 05:38:25.188726 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.188766 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:25.190540 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:25.191401 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:25.191413 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:25.192021 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:25.192611 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.192629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.193734 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.193739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.195537 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.195556 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.196687 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.196695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.196965 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.197771 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:25.197830 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:25.201175 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:25.205329 (+ 4154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:25.205341 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:25.205359 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:25.206092 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:25.206562 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:25.206567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:25.206571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:25.206902 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:25.207507 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:25.207517 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:25.208011 (+ 494us) client_negotiation.cc:769] Sending connection context
0809 05:38:25.208336 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 05:38:25.208760 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":300,"threads_started":1}
I20260809 05:38:25.211578 21140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.194891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.30.137.254:38275)
0809 05:38:25.195653 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.195660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.195680 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:38:25.195745 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.195748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.195832 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.195958 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:25.197735 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.198566 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.200312 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.200772 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.200920 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.204732 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:25.204738 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:25.204742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:25.204785 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:25.208096 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.209029 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.209035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.209038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.209185 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.209814 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.209819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.209822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.210208 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:25.210484 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.210906 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.211135 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":502,"threads_started":1}
I20260809 05:38:25.212442 21135 heartbeater.cc:347] Connected to a master server at 127.30.137.254:38275
I20260809 05:38:25.212783 21135 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:25.213687 21135 heartbeater.cc:511] Master 127.30.137.254:38275 requested a full tablet report, sending...
I20260809 05:38:25.215291 20495 ts_manager.cc:194] Registered new tserver with Master: 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305)
I20260809 05:38:25.216248 20495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:37235
I20260809 05:38:25.228513 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:38:25.235692 21140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.229554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.30.137.254:38275)
0809 05:38:25.229787 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.229796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.230241 (+ 445us) server_negotiation.cc:407] Connection header received
0809 05:38:25.230445 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.230449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.230593 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.230799 (+ 206us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:38:25.231267 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.231990 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.233400 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.233850 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.233951 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.234410 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:38:25.235038 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:38:25.235163 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.235277 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.235349 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:38:25.236088 20757 heartbeater.cc:503] Master 127.30.137.254:38275 was elected leader, sending a full tablet report...
I20260809 05:38:25.236824 20495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:38:25.244531 20495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:38:25.281411 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.257644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41770 (local address 127.30.137.193:44221)
0809 05:38:25.269586 (+ 11942us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.269595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.270955 (+ 1360us) server_negotiation.cc:407] Connection header received
0809 05:38:25.271052 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.271058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.271259 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.271482 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:25.272410 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.274491 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.277647 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.279109 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.279371 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.279568 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.280542 (+ 974us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.280749 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11759,"thread_start_us":11560,"threads_started":1}
I20260809 05:38:25.282097 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.257603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41770)
0809 05:38:25.270082 (+ 12479us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.270102 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:25.270223 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:25.271495 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:25.271502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:25.271532 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:25.272097 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.272116 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.274711 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.274719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.277405 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.277421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.279868 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.279880 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.280163 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.280258 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 05:38:25.280446 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 05:38:25.280685 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12349,"mutex_wait_us":5,"thread_start_us":12148,"threads_started":1}
I20260809 05:38:25.284919 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.257008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:40229 (local address 127.0.0.1:52704)
0809 05:38:25.269590 (+ 12582us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.269653 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:25.269907 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:25.275671 (+ 5764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:25.275676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:25.275704 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:25.276197 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.276211 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.278404 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.278410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.281774 (+ 3364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.281791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.283919 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.283926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.284138 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.284175 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:38:25.284308 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 05:38:25.284465 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12283,"mutex_wait_us":25,"thread_start_us":13135,"threads_started":1}
I20260809 05:38:25.284943 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.257220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.30.137.194:40229)
0809 05:38:25.274693 (+ 17473us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.274704 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.274735 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:38:25.274846 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.274855 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.275203 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.276433 (+ 1230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:25.276676 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.278214 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.281988 (+ 3774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.283658 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.283832 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.283990 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.284363 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.284556 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17212,"thread_start_us":18147,"threads_started":1}
I20260809 05:38:25.286373 20672 tablet_service.cc:1467] Processing CreateTablet for tablet 150557892d9c4ea1b24abc5cccb839b3 (DEFAULT_TABLE table=test-table [id=b2f10ff5e93549288efa62a4f7e461ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:25.289258 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.259700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40305 (local address 127.0.0.1:53930)
0809 05:38:25.280242 (+ 20542us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.280270 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:25.280412 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:25.282290 (+ 1878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:25.282295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:25.282326 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:25.282738 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.282750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.284706 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.284713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.287001 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.287015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.288372 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.288378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.288547 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.288583 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 05:38:25.288697 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 05:38:25.288835 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20369,"thread_start_us":20203,"threads_started":1}
I20260809 05:38:25.289268 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.260543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53930 (local address 127.30.137.195:40305)
0809 05:38:25.279285 (+ 18742us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.279299 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.280403 (+ 1104us) server_negotiation.cc:407] Connection header received
0809 05:38:25.280636 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.280647 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.280921 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.281118 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:25.283075 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.284539 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.287163 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.288139 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.288378 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.288557 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.288755 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.288914 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18444,"thread_start_us":18827,"threads_started":1}
I20260809 05:38:25.289269 20852 tablet_service.cc:1467] Processing CreateTablet for tablet 150557892d9c4ea1b24abc5cccb839b3 (DEFAULT_TABLE table=test-table [id=b2f10ff5e93549288efa62a4f7e461ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:25.290130 20672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150557892d9c4ea1b24abc5cccb839b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:25.290841 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150557892d9c4ea1b24abc5cccb839b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:25.292523 21041 tablet_service.cc:1467] Processing CreateTablet for tablet 150557892d9c4ea1b24abc5cccb839b3 (DEFAULT_TABLE table=test-table [id=b2f10ff5e93549288efa62a4f7e461ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:25.293630 21041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 150557892d9c4ea1b24abc5cccb839b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:25.305363 21163 tablet_bootstrap.cc:492] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Bootstrap starting.
I20260809 05:38:25.307847 21164 tablet_bootstrap.cc:492] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Bootstrap starting.
I20260809 05:38:25.308904 21166 tablet_bootstrap.cc:492] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Bootstrap starting.
I20260809 05:38:25.311177 21163 tablet_bootstrap.cc:654] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:25.311561 21164 tablet_bootstrap.cc:654] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:25.312407 21166 tablet_bootstrap.cc:654] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:25.312630 21163 log.cc:824] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:25.312783 21164 log.cc:824] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:25.313865 21166 log.cc:824] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:25.316499 21164 tablet_bootstrap.cc:492] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: No bootstrap required, opened a new log
I20260809 05:38:25.316601 21163 tablet_bootstrap.cc:492] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: No bootstrap required, opened a new log
I20260809 05:38:25.316838 21164 ts_tablet_manager.cc:1397] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 05:38:25.316895 21163 ts_tablet_manager.cc:1397] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 05:38:25.320502 21166 tablet_bootstrap.cc:492] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: No bootstrap required, opened a new log
I20260809 05:38:25.320794 21166 ts_tablet_manager.cc:1397] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260809 05:38:25.325445 21164 raft_consensus.cc:348] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.325886 21164 raft_consensus.cc:374] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:25.326012 21164 raft_consensus.cc:729] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd22a230f5ee474583c95dc778c59036, State: Initialized, Role: FOLLOWER
I20260809 05:38:25.326803 21164 consensus_queue.cc:260] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.327203 21163 raft_consensus.cc:348] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.327711 21163 raft_consensus.cc:374] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:25.327819 21163 raft_consensus.cc:729] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c8dba24bec4339b99057da06062fc6, State: Initialized, Role: FOLLOWER
I20260809 05:38:25.328258 20951 heartbeater.cc:503] Master 127.30.137.254:38275 was elected leader, sending a full tablet report...
I20260809 05:38:25.329005 21164 ts_tablet_manager.cc:1428] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260809 05:38:25.330114 21163 consensus_queue.cc:260] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.331461 21135 heartbeater.cc:503] Master 127.30.137.254:38275 was elected leader, sending a full tablet report...
I20260809 05:38:25.332034 21163 ts_tablet_manager.cc:1428] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260809 05:38:25.332595 21166 raft_consensus.cc:348] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.333060 21166 raft_consensus.cc:374] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:25.333176 21166 raft_consensus.cc:729] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12342fed55b44c7883d4b64bb5f554d7, State: Initialized, Role: FOLLOWER
I20260809 05:38:25.333819 21166 consensus_queue.cc:260] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.335804 21166 ts_tablet_manager.cc:1428] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260809 05:38:25.436676 21136 tablet.cc:2369] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:25.441869 20952 tablet.cc:2369] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:38:25.452558 20759 tablet.cc:2369] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:38:25.503221 21179 raft_consensus.cc:482] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:38:25.503621 21179 raft_consensus.cc:504] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.505506 21179 leader_election.cc:288] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd22a230f5ee474583c95dc778c59036 (127.30.137.194:40229), 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305)
I20260809 05:38:25.512730 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.506068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:56765 (local address 127.30.137.195:40305)
0809 05:38:25.506338 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.506348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.507153 (+ 805us) server_negotiation.cc:407] Connection header received
0809 05:38:25.507369 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.507372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.507435 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.507534 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:25.508206 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.508933 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.510920 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.511721 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.511919 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.512005 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.512335 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.512462 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:38:25.512774 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.506012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48803 (local address 127.30.137.194:40229)
0809 05:38:25.506244 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.506254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.507054 (+ 800us) server_negotiation.cc:407] Connection header received
0809 05:38:25.507300 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.507305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.507381 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.507500 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:25.508144 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.508885 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.510984 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.511773 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.511960 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.512045 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.512364 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.512507 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:38:25.513155 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.506112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40305 (local address 127.30.137.193:56765)
0809 05:38:25.507029 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.507044 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:25.507210 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:25.507551 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:25.507555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:25.507578 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:25.508021 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.508031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.509099 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.509103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.510756 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.510773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.511975 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.511983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.512158 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.512182 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:38:25.512307 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 05:38:25.512438 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":24,"thread_start_us":474,"threads_started":1}
I20260809 05:38:25.513389 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.505907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:40229 (local address 127.30.137.193:48803)
0809 05:38:25.506915 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.506933 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:25.507134 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:25.507522 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:25.507527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:25.507550 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:25.507984 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.507997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.509029 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.509035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.510817 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:25.510830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.512027 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.512034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:25.512201 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.512224 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:38:25.512334 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 05:38:25.512461 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":67,"thread_start_us":556,"threads_started":1}
I20260809 05:38:25.514263 21066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150557892d9c4ea1b24abc5cccb839b3" candidate_uuid: "12342fed55b44c7883d4b64bb5f554d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c8dba24bec4339b99057da06062fc6" is_pre_election: true
I20260809 05:38:25.514554 20882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150557892d9c4ea1b24abc5cccb839b3" candidate_uuid: "12342fed55b44c7883d4b64bb5f554d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd22a230f5ee474583c95dc778c59036" is_pre_election: true
I20260809 05:38:25.514907 21066 raft_consensus.cc:2454] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12342fed55b44c7883d4b64bb5f554d7 in term 0.
I20260809 05:38:25.515146 20882 raft_consensus.cc:2454] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12342fed55b44c7883d4b64bb5f554d7 in term 0.
I20260809 05:38:25.515661 20605 leader_election.cc:302] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c8dba24bec4339b99057da06062fc6, 12342fed55b44c7883d4b64bb5f554d7; no voters:
I20260809 05:38:25.516139 21179 raft_consensus.cc:2781] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:38:25.516296 21179 raft_consensus.cc:482] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:38:25.516420 21179 raft_consensus.cc:3037] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:25.518827 21179 raft_consensus.cc:504] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.519447 21179 leader_election.cc:288] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [CANDIDATE]: Term 1 election: Requested vote from peers dd22a230f5ee474583c95dc778c59036 (127.30.137.194:40229), 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305)
I20260809 05:38:25.519948 20882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150557892d9c4ea1b24abc5cccb839b3" candidate_uuid: "12342fed55b44c7883d4b64bb5f554d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd22a230f5ee474583c95dc778c59036"
I20260809 05:38:25.519949 21066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "150557892d9c4ea1b24abc5cccb839b3" candidate_uuid: "12342fed55b44c7883d4b64bb5f554d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c8dba24bec4339b99057da06062fc6"
I20260809 05:38:25.520152 20882 raft_consensus.cc:3037] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:25.520172 21066 raft_consensus.cc:3037] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:25.522408 20882 raft_consensus.cc:2454] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12342fed55b44c7883d4b64bb5f554d7 in term 1.
I20260809 05:38:25.522406 21066 raft_consensus.cc:2454] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12342fed55b44c7883d4b64bb5f554d7 in term 1.
I20260809 05:38:25.522863 20605 leader_election.cc:302] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c8dba24bec4339b99057da06062fc6, 12342fed55b44c7883d4b64bb5f554d7; no voters:
I20260809 05:38:25.523218 21179 raft_consensus.cc:2781] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:25.523825 21179 raft_consensus.cc:686] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 LEADER]: Becoming Leader. State: Replica: 12342fed55b44c7883d4b64bb5f554d7, State: Running, Role: LEADER
I20260809 05:38:25.524338 21179 consensus_queue.cc:237] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } }
I20260809 05:38:25.529325 20495 catalog_manager.cc:5495] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "12342fed55b44c7883d4b64bb5f554d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12342fed55b44c7883d4b64bb5f554d7" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 } health_report { overall_health: UNKNOWN } } }
I20260809 05:38:25.569391 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.563273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.30.137.193:44221)
0809 05:38:25.563510 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.563517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.563630 (+ 113us) server_negotiation.cc:407] Connection header received
0809 05:38:25.563908 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.563913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.564009 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.564177 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:38:25.564591 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.565331 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.566948 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.567493 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.567638 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.568218 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:38:25.568792 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:38:25.568972 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.569032 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.569116 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:38:25.578068 20882 raft_consensus.cc:1261] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Refusing update from remote peer 12342fed55b44c7883d4b64bb5f554d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:25.578061 21066 raft_consensus.cc:1261] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Refusing update from remote peer 12342fed55b44c7883d4b64bb5f554d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:25.579128 21184 consensus_queue.cc:1035] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd22a230f5ee474583c95dc778c59036" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:25.579533 21179 consensus_queue.cc:1035] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c8dba24bec4339b99057da06062fc6" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:25.612747 20496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:25.613083 20496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:25.616637 20496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Sending DeleteTablet for 3 replicas of tablet 150557892d9c4ea1b24abc5cccb839b3
I20260809 05:38:25.617817 21041 tablet_service.cc:1514] Processing DeleteTablet for tablet 150557892d9c4ea1b24abc5cccb839b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:25 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:53930
I20260809 05:38:25.617825 20672 tablet_service.cc:1514] Processing DeleteTablet for tablet 150557892d9c4ea1b24abc5cccb839b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:25 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:41770
I20260809 05:38:25.617863 20852 tablet_service.cc:1514] Processing DeleteTablet for tablet 150557892d9c4ea1b24abc5cccb839b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:25 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:52704
I20260809 05:38:25.620345 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 20551
I20260809 05:38:25.627221 21201 tablet_replica.cc:330] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: stopping tablet replica
I20260809 05:38:25.627892 21201 raft_consensus.cc:2229] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:38:25.634851 21201 raft_consensus.cc:2258] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:25.635200 21202 tablet_replica.cc:330] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: stopping tablet replica
I20260809 05:38:25.635848 21203 tablet_replica.cc:330] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: stopping tablet replica
I20260809 05:38:25.636124 21202 raft_consensus.cc:2229] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:25.636453 21203 raft_consensus.cc:2229] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:25.637096 21203 raft_consensus.cc:2258] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:25.637115 21202 raft_consensus.cc:2258] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:25.637419 21201 ts_tablet_manager.cc:1905] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:25.639424 21203 ts_tablet_manager.cc:1905] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:25.640060 21202 ts_tablet_manager.cc:1905] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:25.642336 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.621497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.30.137.193:44221)
0809 05:38:25.621669 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.621675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.621726 (+ 51us) server_negotiation.cc:407] Connection header received
0809 05:38:25.621965 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.621971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.622065 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.622183 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:25.624674 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.625993 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.628771 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.629509 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.629701 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.634433 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:25.634450 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:25.634470 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:25.634530 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:25.638966 (+ 4436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.639809 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.639819 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.639823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.640070 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.640580 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.640585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.640588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.641074 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:25.641399 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.641730 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.641988 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 05:38:25.643427 20749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:25.647447 21203 ts_tablet_manager.cc:1918] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:25.647634 21203 log.cc:1196] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/150557892d9c4ea1b24abc5cccb839b3
I20260809 05:38:25.648252 21203 ts_tablet_manager.cc:1939] T 150557892d9c4ea1b24abc5cccb839b3 P dd22a230f5ee474583c95dc778c59036: Deleting consensus metadata
I20260809 05:38:25.649605 21202 ts_tablet_manager.cc:1918] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:25.649782 21202 log.cc:1196] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/150557892d9c4ea1b24abc5cccb839b3
I20260809 05:38:25.650194 20473 catalog_manager.cc:4841] TS dd22a230f5ee474583c95dc778c59036 (127.30.137.194:40229): tablet 150557892d9c4ea1b24abc5cccb839b3 (table test-table [id=b2f10ff5e93549288efa62a4f7e461ed]) successfully deleted
I20260809 05:38:25.650393 21202 ts_tablet_manager.cc:1939] T 150557892d9c4ea1b24abc5cccb839b3 P 08c8dba24bec4339b99057da06062fc6: Deleting consensus metadata
I20260809 05:38:25.651983 20475 catalog_manager.cc:4841] TS 08c8dba24bec4339b99057da06062fc6 (127.30.137.195:40305): tablet 150557892d9c4ea1b24abc5cccb839b3 (table test-table [id=b2f10ff5e93549288efa62a4f7e461ed]) successfully deleted
I20260809 05:38:25.759321 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 20551
W20260809 05:38:25.785454 20476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.137.193:44221 (error 108)
I20260809 05:38:25.785666 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 20763
I20260809 05:38:25.787621 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.787053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41810)
0809 05:38:25.787267 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.787418 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 05:38:25.787920 20476 catalog_manager.cc:4584] TS 12342fed55b44c7883d4b64bb5f554d7 (127.30.137.193:44221): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 150557892d9c4ea1b24abc5cccb839b3: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
I20260809 05:38:25.800468 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.786573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.30.137.194:40229)
0809 05:38:25.786778 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.786784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.786802 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:38:25.787006 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.787012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.787103 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.787219 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:25.788901 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.789668 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.791128 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.791519 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.791618 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.794320 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:25.794335 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:25.794356 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:25.794395 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:25.797808 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.798515 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.798522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.798524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.798711 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.799127 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.799130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.799132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.799534 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:25.799756 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.800033 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.800198 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:38:25.801160 20942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:25.841917 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.841288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41812)
0809 05:38:25.841554 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.841688 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 05:38:25.843271 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.842858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41816)
0809 05:38:25.843021 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.843115 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 05:38:25.896068 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 20763
I20260809 05:38:25.903955 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.903308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41818)
0809 05:38:25.903551 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.903672 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 05:38:25.905320 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.904827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41828)
0809 05:38:25.905044 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.905153 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 05:38:25.923168 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 20958
I20260809 05:38:25.938172 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.924107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53932 (local address 127.30.137.195:40305)
0809 05:38:25.924326 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:25.924333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:25.924424 (+ 91us) server_negotiation.cc:407] Connection header received
0809 05:38:25.924664 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:25.924668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:25.924749 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:25.924896 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:25.926621 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.927432 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.928816 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:25.929228 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:25.929374 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:25.932329 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:25.932346 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:25.932362 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:25.932405 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:25.935194 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.935977 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.935984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.935987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.936140 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:25.936632 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:25.936636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:25.936638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:25.937119 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:25.937453 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:25.937703 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 05:38:25.937911 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 05:38:25.938951 21129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:25.986196 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.985547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41840)
0809 05:38:25.985830 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.985957 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 05:38:25.987481 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:25.987019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41842)
0809 05:38:25.987247 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:25.987340 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 05:38:26.032624 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 20958
I20260809 05:38:26.061847 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 20452
I20260809 05:38:26.063222 20543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:26.185099 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:26.184380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44221 (local address 127.0.0.1:41850)
0809 05:38:26.184577 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:26.184756 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 05:38:26.185429 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 20452
I20260809 05:38:26.218569 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:38275
--webserver_interface=127.30.137.254
--webserver_port=43599
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:38275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:26.577982 21211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:26.578390 21211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:26.578563 21211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:26.586158 21211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:26.586318 21211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:26.586426 21211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:26.586534 21211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:26.586621 21211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:26.596674 21211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:38275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:38275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=43599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:26.598088 21211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:26.599444 21211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:26.611398 21218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:26.611445 21222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:26.611481 21219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:26.612753 21211 server_base.cc:1034] running on GCE node
I20260809 05:38:26.613548 21211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:26.615087 21211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:26.616323 21211 hybrid_clock.cc:648] HybridClock initialized: now 1786253906616282 us; error 49 us; skew 500 ppm
I20260809 05:38:26.616660 21211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:26 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:26.619868 21211 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:26.621254 21211 webserver.cc:460] Webserver started at http://127.30.137.254:43599/ using document root <none> and password file <none>
I20260809 05:38:26.621886 21211 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:26.621986 21211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:26.627274 21211 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 05:38:26.632763 21231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:26.634178 21211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 05:38:26.634388 21211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "d111d9a125d249f9b712fe1819fb1f38"
format_stamp: "Formatted at 2026-08-09 05:38:19 on dist-test-slave-3kk6"
server_key: "d41f6b7d6d7d147f5d70d13e98f1d3bf"
server_key_iv: "1109dcd15c0a135ca653c47cea921f1a"
server_key_version: "encryptionkey@0"
I20260809 05:38:26.635542 21211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:26.659638 21211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:26.664394 21211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:26.664755 21211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:26.684898 21211 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:38275
I20260809 05:38:26.684921 21304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:38275 every 8 connection(s)
I20260809 05:38:26.686486 21211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:26.690837 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 21211
I20260809 05:38:26.691936 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:44221
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:38275
--webserver_port=45005
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:38:26.694926 21305 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:38:26.700333 21305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap starting.
I20260809 05:38:26.722627 21305 log.cc:824] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:26.740489 21305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:38:26.741109 21305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38: Bootstrap complete.
I20260809 05:38:26.750739 21305 raft_consensus.cc:348] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:26.751724 21305 raft_consensus.cc:729] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d111d9a125d249f9b712fe1819fb1f38, State: Initialized, Role: FOLLOWER
I20260809 05:38:26.752404 21305 consensus_queue.cc:260] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:26.752655 21305 raft_consensus.cc:388] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:26.752815 21305 raft_consensus.cc:482] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:26.753017 21305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 2 FOLLOWER]: Advancing to term 3
I20260809 05:38:26.757277 21305 raft_consensus.cc:504] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:26.757812 21305 leader_election.cc:302] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d111d9a125d249f9b712fe1819fb1f38; no voters:
I20260809 05:38:26.758529 21305 leader_election.cc:288] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 05:38:26.758716 21309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 05:38:26.759979 21309 raft_consensus.cc:686] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [term 3 LEADER]: Becoming Leader. State: Replica: d111d9a125d249f9b712fe1819fb1f38, State: Running, Role: LEADER
I20260809 05:38:26.760547 21309 consensus_queue.cc:237] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } }
I20260809 05:38:26.760924 21305 sys_catalog.cc:564] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:26.763412 21311 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d111d9a125d249f9b712fe1819fb1f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } } }
I20260809 05:38:26.763865 21311 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:26.766251 21313 sys_catalog.cc:455] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d111d9a125d249f9b712fe1819fb1f38. Latest consensus state: current_term: 3 leader_uuid: "d111d9a125d249f9b712fe1819fb1f38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d111d9a125d249f9b712fe1819fb1f38" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 38275 } } }
I20260809 05:38:26.766803 21313 sys_catalog.cc:458] T 00000000000000000000000000000000 P d111d9a125d249f9b712fe1819fb1f38 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:26.768857 21320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:26.780187 21320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:26.782259 21320 catalog_manager.cc:1259] Loaded cluster ID: 9fce34ef1a524b16bd7d1715151b8761
I20260809 05:38:26.782413 21320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 05:38:26.785296 21320 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f96637699ad google::LogMessage::Fail() at ??:0
@ 0x7f966376c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f96637694de google::LogMessage::Flush() at ??:0
I20260809 05:38:26.794837 21332 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f966376aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f967919c512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f9679199976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9679221610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9679221439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f96663b5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f96663b891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f96663b8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9666393eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f966966d6db start_thread at ??:0
@ 0x7f966260361f clone at ??:0
W20260809 05:38:27.116947 21307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:27.117262 21307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:27.117372 21307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:27.124913 21307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:27.125052 21307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:27.125187 21307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:38:27.135346 21307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:44221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=45005
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:27.136663 21307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:27.138021 21307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:27.152231 21362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:27.152164 21364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:27.152124 21366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:27.153702 21307 server_base.cc:1034] running on GCE node
I20260809 05:38:27.154410 21307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:27.155452 21307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:27.156668 21307 hybrid_clock.cc:648] HybridClock initialized: now 1786253907156626 us; error 39 us; skew 500 ppm
I20260809 05:38:27.156971 21307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:27 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:27.160264 21307 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:38:27.161898 21307 webserver.cc:460] Webserver started at http://127.30.137.193:45005/ using document root <none> and password file <none>
I20260809 05:38:27.162570 21307 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:27.162732 21307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:27.168231 21307 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 05:38:27.171417 21377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:27.172850 21307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 05:38:27.173036 21307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "12342fed55b44c7883d4b64bb5f554d7"
format_stamp: "Formatted at 2026-08-09 05:38:20 on dist-test-slave-3kk6"
server_key: "72b5098cb84ddc3f3734d27da9c71402"
server_key_iv: "afe68270e468f5944c1849d1aa8e29fa"
server_key_version: "encryptionkey@0"
I20260809 05:38:27.174068 21307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:27.189332 21307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:27.193886 21307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:27.194242 21307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:27.195448 21307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:27.197939 21385 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260809 05:38:27.203599 21385 ts_tablet_manager.cc:1852] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 05:38:27.204660 21385 ts_tablet_manager.cc:1905] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:27.212555 21385 ts_tablet_manager.cc:1918] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:27.212752 21385 log.cc:1196] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/150557892d9c4ea1b24abc5cccb839b3
I20260809 05:38:27.213217 21385 ts_tablet_manager.cc:1939] T 150557892d9c4ea1b24abc5cccb839b3 P 12342fed55b44c7883d4b64bb5f554d7: Deleting consensus metadata
I20260809 05:38:27.213604 21307 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 05:38:27.213770 21307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260809 05:38:27.213925 21307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:27.214028 21307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:27.250516 21307 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44221
I20260809 05:38:27.250627 21544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44221 every 8 connection(s)
I20260809 05:38:27.252074 21307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:38:27.256960 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 21307
I20260809 05:38:27.257324 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:27.255383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:51589)
0809 05:38:27.256133 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:27.256408 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":295,"threads_started":1}
I20260809 05:38:27.257678 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:40229
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:38275
--webserver_port=37359
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:27.258085 21545 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:38275 (0 consecutive failures): Network error: Failed to ping master at 127.30.137.254:38275: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
I20260809 05:38:27.259033 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:27.258644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:50205)
0809 05:38:27.258834 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:27.258931 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 05:38:27.260087 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:27.259598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:53927)
0809 05:38:27.259783 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:27.259905 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 05:38:27.260370 21545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 05:38:27.608151 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:27.608474 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:27.608574 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:27.616078 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:27.616212 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:27.616420 21552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:38:27.626487 21552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:40229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=37359
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:27.627969 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:27.629375 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:27.643860 21561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:27.643800 21558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:27.643913 21559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:27.644824 21552 server_base.cc:1034] running on GCE node
I20260809 05:38:27.645525 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:27.646509 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:27.647809 21552 hybrid_clock.cc:648] HybridClock initialized: now 1786253907647728 us; error 77 us; skew 500 ppm
I20260809 05:38:27.648160 21552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:27 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:27.656067 21552 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:38:27.657806 21552 webserver.cc:460] Webserver started at http://127.30.137.194:37359/ using document root <none> and password file <none>
I20260809 05:38:27.658610 21552 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:27.658838 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:27.664738 21552 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 05:38:27.667917 21573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:27.669513 21552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:27.669720 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "dd22a230f5ee474583c95dc778c59036"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "07293913c20187f2aed42c0e93656a96"
server_key_iv: "64b157d4eb0dc9267b95b112be83c471"
server_key_version: "encryptionkey@0"
I20260809 05:38:27.670961 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:27.685426 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:27.690264 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:27.690593 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:27.691717 21552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:27.693176 21552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:27.693305 21552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:27.693451 21552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:27.693548 21552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:27.729393 21552 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:40229
I20260809 05:38:27.729502 21735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:40229 every 8 connection(s)
I20260809 05:38:27.731127 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:38:27.736011 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:27.734165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:56615)
0809 05:38:27.734972 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:27.735200 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":296,"threads_started":1}
W20260809 05:38:27.736670 21736 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:38275 (0 consecutive failures): Network error: Failed to ping master at 127.30.137.254:38275: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
I20260809 05:38:27.737660 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:27.737342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:49961)
0809 05:38:27.737489 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:27.737561 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 05:38:27.738526 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:27.738214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:40897)
0809 05:38:27.738343 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:27.738419 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 05:38:27.738799 21736 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 05:38:27.739892 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 21552
I20260809 05:38:27.740577 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:40305
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:38275
--webserver_port=45717
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:28.090036 21741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:28.090363 21741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:28.090529 21741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:28.101569 21741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:28.101768 21741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:28.102043 21741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:38:28.113343 21741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:40305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=45717
--tserver_master_addrs=127.30.137.254:38275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:28.114827 21741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:28.116233 21741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:28.130730 21747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:28.130671 21753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:28.130730 21750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:28.131708 21741 server_base.cc:1034] running on GCE node
I20260809 05:38:28.132380 21741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:28.133426 21741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:28.134714 21741 hybrid_clock.cc:648] HybridClock initialized: now 1786253908134617 us; error 78 us; skew 500 ppm
I20260809 05:38:28.135077 21741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:28 dist-test-slave-3kk6 krb5kdc[19646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:28.138514 21741 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:38:28.140177 21741 webserver.cc:460] Webserver started at http://127.30.137.195:45717/ using document root <none> and password file <none>
I20260809 05:38:28.140936 21741 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:28.141099 21741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:28.147125 21741 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:38:28.150029 21762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:28.151582 21741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:28.151796 21741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "08c8dba24bec4339b99057da06062fc6"
format_stamp: "Formatted at 2026-08-09 05:38:21 on dist-test-slave-3kk6"
server_key: "ec9fe12faa149f289a787908d55cfcb5"
server_key_iv: "39b49561381ea06485f15c0113a3f847"
server_key_version: "encryptionkey@0"
I20260809 05:38:28.152902 21741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:28.170527 21741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:28.174950 21741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:28.175196 21741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:28.176373 21741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:28.177842 21741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:28.177970 21741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:28.178086 21741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:28.178158 21741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:28.215060 21741 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40305
I20260809 05:38:28.215147 21929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40305 every 8 connection(s)
I20260809 05:38:28.216701 21741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:38:28.221743 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:28.219864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:53621)
0809 05:38:28.220635 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:28.220864 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":313,"threads_started":1}
I20260809 05:38:28.221812 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 21741
W20260809 05:38:28.222618 21931 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:38275 (0 consecutive failures): Network error: Failed to ping master at 127.30.137.254:38275: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
I20260809 05:38:28.223503 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:28.223158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:37045)
0809 05:38:28.223317 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:28.223396 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 05:38:28.224462 21934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:28.224140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:43747)
0809 05:38:28.224276 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:28.224353 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 05:38:28.224723 21931 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 05:38:28.261744 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:28.260928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:45691)
0809 05:38:28.261406 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:28.261500 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":226,"threads_started":1}
I20260809 05:38:28.740178 21937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:28.739295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:36589)
0809 05:38:28.739820 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:28.739953 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":248,"threads_started":1}
I20260809 05:38:29.226125 21939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:29.225292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:42427)
0809 05:38:29.225761 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:29.225895 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":236,"threads_started":1}
I20260809 05:38:29.263578 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:29.262859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:56057)
0809 05:38:29.263310 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:29.263415 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
I20260809 05:38:29.742019 21943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:29.741214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:50735)
0809 05:38:29.741745 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:29.741848 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":263,"threads_started":1}
I20260809 05:38:30.228045 21944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:30.227191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:41539)
0809 05:38:30.227768 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:30.227866 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":294,"threads_started":1}
I20260809 05:38:30.265424 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:30.264579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:48427)
0809 05:38:30.265117 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:30.265226 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":279,"threads_started":1}
I20260809 05:38:30.743764 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:30.743014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:47515)
0809 05:38:30.743515 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:30.743594 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":275,"threads_started":1}
I20260809 05:38:31.229715 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:31.228974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:33117)
0809 05:38:31.229459 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:31.229542 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":260,"threads_started":1}
I20260809 05:38:31.267144 21953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:31.266309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:51709)
0809 05:38:31.266885 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:31.266976 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":249,"threads_started":1}
I20260809 05:38:31.745592 21955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:31.744743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:51293)
0809 05:38:31.745270 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:31.745370 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":249,"threads_started":1}
I20260809 05:38:32.231679 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:32.230891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:57803)
0809 05:38:32.231371 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:32.231461 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":232,"threads_started":1}
I20260809 05:38:32.268999 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:32.268167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:59923)
0809 05:38:32.268641 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:32.268750 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":241,"threads_started":1}
I20260809 05:38:32.747463 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:32.746604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:34071)
0809 05:38:32.747152 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:32.747265 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":285,"threads_started":1}
I20260809 05:38:33.233539 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:33.232652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:53621)
0809 05:38:33.233242 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:33.233355 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":341,"threads_started":1}
I20260809 05:38:33.270949 21961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:33.269978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:45901)
0809 05:38:33.270543 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:33.270693 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":268,"threads_started":1}
I20260809 05:38:33.749191 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:33.748428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:33561)
0809 05:38:33.748930 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:33.749026 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":265,"threads_started":1}
I20260809 05:38:34.235493 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:34.234638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:58627)
0809 05:38:34.235220 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:34.235323 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":271,"threads_started":1}
I20260809 05:38:34.273134 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:34.272067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:59433)
0809 05:38:34.272739 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:34.272865 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":288,"threads_started":1}
I20260809 05:38:34.751154 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:34.750227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:48691)
0809 05:38:34.750831 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:34.750923 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":243,"threads_started":1}
I20260809 05:38:35.237534 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:35.236564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:53513)
0809 05:38:35.237179 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:35.237279 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":270,"threads_started":1}
I20260809 05:38:35.274928 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:35.274135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:43609)
0809 05:38:35.274604 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:35.274749 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":258,"threads_started":1}
I20260809 05:38:35.753036 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:35.752276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:40647)
0809 05:38:35.752768 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:35.752863 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":254,"threads_started":1}
I20260809 05:38:36.239665 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:36.238732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:42531)
0809 05:38:36.239396 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:36.239492 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":394,"threads_started":1}
I20260809 05:38:36.276880 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:36.275940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:51035)
0809 05:38:36.276555 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:36.276655 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":327,"threads_started":1}
I20260809 05:38:36.754828 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:36.754013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:43785)
0809 05:38:36.754538 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:36.754623 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
I20260809 05:38:37.241516 21976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:37.240696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:44987)
0809 05:38:37.241229 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:37.241324 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":270,"threads_started":1}
I20260809 05:38:37.278772 21978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:37.277935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:37089)
0809 05:38:37.278431 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:37.278538 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":251,"threads_started":1}
I20260809 05:38:37.756879 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:37.755947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:45213)
0809 05:38:37.756584 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:37.756685 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":354,"threads_started":1}
I20260809 05:38:38.243495 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:38.242592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:45127)
0809 05:38:38.243240 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:38.243321 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":416,"threads_started":1}
I20260809 05:38:38.280422 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:38.279676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:35167)
0809 05:38:38.280133 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:38.280212 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":257,"threads_started":1}
I20260809 05:38:38.758963 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:38.757994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:46217)
0809 05:38:38.758566 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:38.758717 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":315,"threads_started":1}
I20260809 05:38:39.245246 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:39.244491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:51513)
0809 05:38:39.244964 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:39.245058 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":237,"threads_started":1}
I20260809 05:38:39.282107 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:39.281356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:42923)
0809 05:38:39.281847 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:39.281942 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":267,"threads_started":1}
I20260809 05:38:39.760711 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:39.759957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:51569)
0809 05:38:39.760453 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:39.760548 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
I20260809 05:38:40.247486 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:40.246441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:48801)
0809 05:38:40.247173 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:40.247278 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":427,"threads_started":1}
I20260809 05:38:40.284116 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:40.283183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:53385)
0809 05:38:40.283783 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:40.283901 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":307,"threads_started":1}
I20260809 05:38:40.763007 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:40.761930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:59509)
0809 05:38:40.762569 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:40.762735 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":337,"threads_started":1}
I20260809 05:38:41.249516 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:41.248623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:34443)
0809 05:38:41.249193 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:41.249310 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":274,"threads_started":1}
I20260809 05:38:41.286056 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:41.285234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:46809)
0809 05:38:41.285765 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:41.285866 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":279,"threads_started":1}
I20260809 05:38:41.764846 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:41.763976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:60571)
0809 05:38:41.764532 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:41.764627 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":270,"threads_started":1}
I20260809 05:38:42.251626 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:42.250606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:38747)
0809 05:38:42.251203 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:42.251351 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":298,"threads_started":1}
I20260809 05:38:42.288139 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:42.287343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.193:42661)
0809 05:38:42.287846 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:42.287950 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":249,"threads_started":1}
I20260809 05:38:42.767035 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:42.766162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.194:59565)
0809 05:38:42.766698 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:42.766839 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":253,"threads_started":1}
I20260809 05:38:43.230862 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 21307
I20260809 05:38:43.250145 21996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:43.232523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.30.137.193:44221)
0809 05:38:43.233357 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:43.233379 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:43.233445 (+ 66us) server_negotiation.cc:407] Connection header received
0809 05:38:43.235699 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:43.235709 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:43.236155 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:43.236470 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:43.237963 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:43.239295 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:43.240539 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:43.241212 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:43.241355 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:43.244158 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:43.244166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:43.244182 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:43.244224 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:43.247234 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:43.247917 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:43.247923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:43.247929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:43.248085 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:43.248435 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:43.248438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:43.248440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:43.248958 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:43.249185 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:43.249531 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 05:38:43.249853 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":352,"threads_started":1}
I20260809 05:38:43.251592 21537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:43.254077 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:43.253065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:38275 (local address 127.30.137.195:37029)
0809 05:38:43.253665 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:43.253815 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:38275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":285,"threads_started":1}
I20260809 05:38:43.339483 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 21307
I20260809 05:38:43.363343 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 21552
I20260809 05:38:43.381762 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:43.364330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37734 (local address 127.30.137.194:40229)
0809 05:38:43.364885 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:43.364899 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:43.364938 (+ 39us) server_negotiation.cc:407] Connection header received
0809 05:38:43.366588 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:43.366598 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:43.367123 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:43.367426 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:43.368928 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:43.370150 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:43.371554 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:43.372241 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:43.372400 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:43.375190 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:43.375198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:43.375212 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:43.375255 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:43.378482 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:43.379366 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:43.379373 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:43.379380 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:43.379547 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:43.379902 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:43.379905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:43.379907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:43.380431 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:43.380670 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:43.381050 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 05:38:43.381477 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":255,"threads_started":1}
I20260809 05:38:43.383177 21726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:43.468604 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 21552
I20260809 05:38:43.492256 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 21741
I20260809 05:38:43.511868 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:43.493369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.30.137.195:40305)
0809 05:38:43.494007 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:43.494023 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:43.494066 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:38:43.495812 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:43.495821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:43.496300 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:43.496667 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:43.498340 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:43.499760 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:43.501155 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:43.501825 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:43.502045 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:43.505120 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:43.505129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:43.505144 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:43.505195 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:43.508396 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:43.509115 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:43.509123 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:43.509129 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:43.509313 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:43.509720 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:43.509726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:43.509730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:43.510268 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:43.510627 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:43.511097 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 05:38:43.511533 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":304,"threads_started":1}
I20260809 05:38:43.513252 21923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:43.598398 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 21741
2026-08-09T05:38:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25719 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:38:43 dist-test-slave-3kk6 krb5kdc[22012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:38:43 dist-test-slave-3kk6 krb5kdc[22012](info): set up 2 sockets
Aug 09 05:38:43 dist-test-slave-3kk6 krb5kdc[22012](info): commencing operation
krb5kdc: starting...
W20260809 05:38:44.925218 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:38:45 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:38:45.241822 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34445
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:34445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:45.594734 22031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:45.595084 22031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:45.595178 22031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:45.602519 22031 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 05:38:45.602630 22031 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 05:38:45.602746 22031 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 05:38:45.602815 22031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:38:45.602874 22031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:45.602946 22031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:45.602994 22031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:38:45.603044 22031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:38:45.613346 22031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:34445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:45.614624 22031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:45.615988 22031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:45.628496 22042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:45.628517 22044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:45.628571 22040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:45.628863 22031 server_base.cc:1034] running on GCE node
I20260809 05:38:45.630072 22031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:45.631708 22031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:45.632972 22031 hybrid_clock.cc:648] HybridClock initialized: now 1786253925632907 us; error 45 us; skew 500 ppm
I20260809 05:38:45.633330 22031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:45 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:45.636643 22031 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:38:45.638233 22031 webserver.cc:460] Webserver started at http://127.30.137.254:37405/ using document root <none> and password file <none>
I20260809 05:38:45.639042 22031 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:45.639189 22031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:45.639528 22031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:45.642632 22031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "a6ce0cada05b43fe9b6b37ee413c6585"
format_stamp: "Formatted at 2026-08-09 05:38:45 on dist-test-slave-3kk6"
server_key: "f97fdffc6013344f99b8f612d9b73c6e"
server_key_iv: "0979adf0b3918a894c0b3e6f4da1aa90"
server_key_version: "encryptionkey@0"
I20260809 05:38:45.643586 22031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "a6ce0cada05b43fe9b6b37ee413c6585"
format_stamp: "Formatted at 2026-08-09 05:38:45 on dist-test-slave-3kk6"
server_key: "f97fdffc6013344f99b8f612d9b73c6e"
server_key_iv: "0979adf0b3918a894c0b3e6f4da1aa90"
server_key_version: "encryptionkey@0"
I20260809 05:38:45.649166 22031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:38:45.652712 22055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:45.654094 22031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:38:45.654296 22031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "a6ce0cada05b43fe9b6b37ee413c6585"
format_stamp: "Formatted at 2026-08-09 05:38:45 on dist-test-slave-3kk6"
server_key: "f97fdffc6013344f99b8f612d9b73c6e"
server_key_iv: "0979adf0b3918a894c0b3e6f4da1aa90"
server_key_version: "encryptionkey@0"
I20260809 05:38:45.654544 22031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:45.688179 22031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:45.692545 22031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:45.692862 22031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:45.713090 22031 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:34445
I20260809 05:38:45.713069 22128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:34445 every 8 connection(s)
I20260809 05:38:45.714598 22031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:38:45.718433 22131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:45.722853 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 22031
I20260809 05:38:45.723101 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:38:45.723955 31271 external_mini_cluster.cc:1428] Setting key d355f5d64a391e65b392dc38f39d1644
I20260809 05:38:45.730785 22131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap starting.
I20260809 05:38:45.734153 22131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:45.735292 22131 log.cc:824] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:45.738245 22131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: No bootstrap required, opened a new log
Aug 09 05:38:45 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:45.747881 22131 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:38:45.748315 22131 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:45.748461 22131 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ce0cada05b43fe9b6b37ee413c6585, State: Initialized, Role: FOLLOWER
I20260809 05:38:45.749094 22131 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:38:45.749373 22131 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:38:45.749382 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:45.726333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.30.137.254:34445)
0809 05:38:45.727145 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:45.727157 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:45.727200 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:38:45.729460 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:45.729485 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:45.729896 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:45.730304 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:45.732053 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:45.733452 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:45.734957 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:45.735728 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:45.735970 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:45.740972 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:45.740979 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:45.740993 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:45.741042 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:45.744480 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:45.745413 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:45.745420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:45.745427 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:45.745613 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:45.746051 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:45.746055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:45.746056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:45.746702 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:45.746966 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:45.747473 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 05:38:45.747885 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":385,"threads_started":1}
I20260809 05:38:45.749574 22131 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:38:45.749816 22131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:45.752444 22131 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:38:45.752974 22131 leader_election.cc:302] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6ce0cada05b43fe9b6b37ee413c6585; no voters:
I20260809 05:38:45.753603 22131 leader_election.cc:288] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:38:45.753746 22136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:45.754504 22136 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 LEADER]: Becoming Leader. State: Replica: a6ce0cada05b43fe9b6b37ee413c6585, State: Running, Role: LEADER
I20260809 05:38:45.755110 22136 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:38:45.755812 22131 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:38:45.758270 22139 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6ce0cada05b43fe9b6b37ee413c6585. Latest consensus state: current_term: 1 leader_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } } }
I20260809 05:38:45.758796 22139 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:45.760074 22138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } } }
I20260809 05:38:45.760421 22138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: This master's current role is: LEADER
I20260809 05:38:45.762013 22146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:38:45.767030 22146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:38:45.775996 22146 catalog_manager.cc:1347] Generated new cluster ID: fa4302127733423d95194e451793f5db
I20260809 05:38:45.776132 22146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:38:45.793224 22146 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:38:45.794607 22146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:38:45.805615 22146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Generated new TSK 0
I20260809 05:38:45.806540 22146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:38:45.880630 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:34445
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:46.232066 22165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:46.232458 22165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:46.232640 22165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:46.239809 22165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:46.239964 22165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:46.240171 22165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:38:46.250048 22165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:46.251508 22165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:46.252857 22165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:46.266440 22173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:46.266515 22176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:46.266477 22179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:46.267740 22165 server_base.cc:1034] running on GCE node
I20260809 05:38:46.268364 22165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:46.269236 22165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:46.270447 22165 hybrid_clock.cc:648] HybridClock initialized: now 1786253926270403 us; error 33 us; skew 500 ppm
I20260809 05:38:46.270793 22165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:46 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:46.273960 22165 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:38:46.275488 22165 webserver.cc:460] Webserver started at http://127.30.137.193:35461/ using document root <none> and password file <none>
I20260809 05:38:46.276185 22165 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:46.276326 22165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:46.276648 22165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:46.279712 22165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "a1457f307f3045c4ad8ea72d8ae584f8"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "6c51c439ac1d289e9c6237d1e8f02b6f"
server_key_iv: "598a54630d63ae9204712e080ca4a0f3"
server_key_version: "encryptionkey@0"
I20260809 05:38:46.280530 22165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "a1457f307f3045c4ad8ea72d8ae584f8"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "6c51c439ac1d289e9c6237d1e8f02b6f"
server_key_iv: "598a54630d63ae9204712e080ca4a0f3"
server_key_version: "encryptionkey@0"
I20260809 05:38:46.285832 22165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:38:46.289315 22190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:46.290778 22165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:46.290967 22165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "a1457f307f3045c4ad8ea72d8ae584f8"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "6c51c439ac1d289e9c6237d1e8f02b6f"
server_key_iv: "598a54630d63ae9204712e080ca4a0f3"
server_key_version: "encryptionkey@0"
I20260809 05:38:46.291188 22165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:46.310218 22165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:46.314534 22165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:46.314872 22165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:46.315860 22165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:46.317236 22165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:46.317351 22165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:46.317502 22165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:46.317576 22165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:46.352214 22165 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:33581
I20260809 05:38:46.352308 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:33581 every 8 connection(s)
I20260809 05:38:46.353565 22165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:38:46.363648 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 22165
I20260809 05:38:46.363888 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:38:46.364583 31271 external_mini_cluster.cc:1428] Setting key 467bee13863702b4b6481dfbc2da0145
Aug 09 05:38:46 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:46.374818 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:46.356321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:53537 (local address 127.30.137.254:34445)
0809 05:38:46.356833 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:46.356841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:46.357757 (+ 916us) server_negotiation.cc:407] Connection header received
0809 05:38:46.359840 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:46.359846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:46.359949 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:46.360076 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:46.361706 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.362471 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.364196 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.364646 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.364785 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:46.368890 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:46.368895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:46.368899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:46.368932 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:46.371640 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:46.372406 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:46.372414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:46.372422 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:46.372538 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:46.372989 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:46.372994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:46.372999 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:46.373268 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:46.373437 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:46.374176 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 05:38:46.374426 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":272,"threads_started":1}
I20260809 05:38:46.375763 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:46.356826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:53537)
0809 05:38:46.357571 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:46.357615 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:46.359551 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:46.360331 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:46.360339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:46.360921 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:46.361511 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:46.361528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.362619 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.362624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:46.364051 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:46.364064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.364894 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.364903 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:46.365155 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:46.365949 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:46.365986 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:46.368658 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:46.371815 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:46.371825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:46.371848 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:46.372272 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:46.372654 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:46.372659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:46.372662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:46.372865 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:46.373409 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:46.373417 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:46.373767 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 05:38:46.374051 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 05:38:46.374597 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":287,"threads_started":1}
I20260809 05:38:46.377462 22354 heartbeater.cc:347] Connected to a master server at 127.30.137.254:34445
I20260809 05:38:46.377774 22354 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:46.378530 22354 heartbeater.cc:511] Master 127.30.137.254:34445 requested a full tablet report, sending...
I20260809 05:38:46.380690 22081 ts_manager.cc:194] Registered new tserver with Master: a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193:33581)
I20260809 05:38:46.382176 22081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:53537
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:38:46.440037 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:34445
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:46.788502 22361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:46.788875 22361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:46.789064 22361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:46.796634 22361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:46.796793 22361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:46.796983 22361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:38:46.806725 22361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:46.808094 22361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:46.809458 22361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:46.823101 22370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:46.823213 22369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:46.823266 22372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:46.824258 22361 server_base.cc:1034] running on GCE node
I20260809 05:38:46.824895 22361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:46.825881 22361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:46.827105 22361 hybrid_clock.cc:648] HybridClock initialized: now 1786253926827072 us; error 49 us; skew 500 ppm
I20260809 05:38:46.827447 22361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:46 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:46.833407 22361 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:38:46.835127 22361 webserver.cc:460] Webserver started at http://127.30.137.194:42741/ using document root <none> and password file <none>
I20260809 05:38:46.835842 22361 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:46.835996 22361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:46.836333 22361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:46.839408 22361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "7c3e2f1a5a324cb985896d007d0e6a32"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "1b362ec57a26b3abca7462012c3a2210"
server_key_iv: "5adc0fb1815ad435fe0c44e17b91ecc4"
server_key_version: "encryptionkey@0"
I20260809 05:38:46.840215 22361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "7c3e2f1a5a324cb985896d007d0e6a32"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "1b362ec57a26b3abca7462012c3a2210"
server_key_iv: "5adc0fb1815ad435fe0c44e17b91ecc4"
server_key_version: "encryptionkey@0"
I20260809 05:38:46.845392 22361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:38:46.848759 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:46.850051 22361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 05:38:46.850230 22361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "7c3e2f1a5a324cb985896d007d0e6a32"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "1b362ec57a26b3abca7462012c3a2210"
server_key_iv: "5adc0fb1815ad435fe0c44e17b91ecc4"
server_key_version: "encryptionkey@0"
I20260809 05:38:46.850457 22361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:46.872663 22361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:46.876976 22361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:46.877296 22361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:46.878274 22361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:46.879664 22361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:46.879792 22361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:46.879945 22361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:46.880020 22361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:46.916590 22361 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46597
I20260809 05:38:46.916651 22542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46597 every 8 connection(s)
I20260809 05:38:46.917963 22361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:38:46.921589 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 22361
I20260809 05:38:46.921846 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:38:46.922525 31271 external_mini_cluster.cc:1428] Setting key 311c04ef500c9981e05e482b0610083a
Aug 09 05:38:46 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253926, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:46.941452 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:46.920897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:50215 (local address 127.30.137.254:34445)
0809 05:38:46.921439 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:46.921446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:46.922633 (+ 1187us) server_negotiation.cc:407] Connection header received
0809 05:38:46.924718 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:46.924723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:46.924802 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:46.924892 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:46.927136 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.927864 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.930393 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.930919 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.931028 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:46.935490 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:46.935496 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:46.935499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:46.935532 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:46.938191 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:46.939047 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:46.939051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:46.939054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:46.939183 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:46.939639 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:46.939649 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:46.939651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:46.939917 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:46.940075 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:46.940806 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 05:38:46.941056 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260809 05:38:46.942489 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:46.921518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:50215)
0809 05:38:46.922428 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:46.922477 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:46.924426 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:46.925184 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:46.925196 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:46.926020 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:46.926929 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:46.926958 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.928021 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.928029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:46.930218 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:46.930237 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:46.931154 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:46.931164 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:46.931508 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:46.932342 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:46.932378 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:46.935243 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:46.938339 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:46.938350 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:46.938367 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:46.938906 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:46.939304 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:46.939310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:46.939314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:46.939513 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:46.940051 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:46.940059 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:46.940415 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 05:38:46.940691 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 05:38:46.941252 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":372,"threads_started":1}
I20260809 05:38:46.944190 22543 heartbeater.cc:347] Connected to a master server at 127.30.137.254:34445
I20260809 05:38:46.944525 22543 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:46.945263 22543 heartbeater.cc:511] Master 127.30.137.254:34445 requested a full tablet report, sending...
I20260809 05:38:46.946916 22081 ts_manager.cc:194] Registered new tserver with Master: 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194:46597)
I20260809 05:38:46.947893 22081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:50215
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:38:46.993990 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:34445
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:38:47.346997 22553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:38:47.347287 22553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:38:47.347492 22553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:38:47.357148 22553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:38:47.357304 22553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:38:47.357493 22553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:38:47.368198 22553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:38:47.369410 22553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:38:47.370640 22553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:38:47.385113 22559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:47.385061 22562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:38:47.385169 22560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:38:47.386013 22354 heartbeater.cc:503] Master 127.30.137.254:34445 was elected leader, sending a full tablet report...
I20260809 05:38:47.391319 22553 server_base.cc:1034] running on GCE node
I20260809 05:38:47.392025 22553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:38:47.393007 22553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:38:47.394251 22553 hybrid_clock.cc:648] HybridClock initialized: now 1786253927394196 us; error 30 us; skew 500 ppm
I20260809 05:38:47.394585 22553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:38:47 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:38:47.397971 22553 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:38:47.399986 22553 webserver.cc:460] Webserver started at http://127.30.137.195:40685/ using document root <none> and password file <none>
I20260809 05:38:47.400718 22553 fs_manager.cc:359] Metadata directory not provided
I20260809 05:38:47.400869 22553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:38:47.401275 22553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:38:47.404551 22553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "18bc2c169dba426b809034133794a8e5"
format_stamp: "Formatted at 2026-08-09 05:38:47 on dist-test-slave-3kk6"
server_key: "2133b753e7458c73e2187894cf70b7fa"
server_key_iv: "8d632ed6c3e187a3624e0c86d96a4fa6"
server_key_version: "encryptionkey@0"
I20260809 05:38:47.405334 22553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "18bc2c169dba426b809034133794a8e5"
format_stamp: "Formatted at 2026-08-09 05:38:47 on dist-test-slave-3kk6"
server_key: "2133b753e7458c73e2187894cf70b7fa"
server_key_iv: "8d632ed6c3e187a3624e0c86d96a4fa6"
server_key_version: "encryptionkey@0"
I20260809 05:38:47.410614 22553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:38:47.414075 22575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:47.415407 22553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:38:47.415575 22553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "18bc2c169dba426b809034133794a8e5"
format_stamp: "Formatted at 2026-08-09 05:38:47 on dist-test-slave-3kk6"
server_key: "2133b753e7458c73e2187894cf70b7fa"
server_key_iv: "8d632ed6c3e187a3624e0c86d96a4fa6"
server_key_version: "encryptionkey@0"
I20260809 05:38:47.415787 22553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:38:47.439132 22553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:38:47.443745 22553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:38:47.444070 22553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:38:47.445112 22553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:38:47.446488 22553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:38:47.446568 22553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:47.446759 22553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:38:47.446817 22553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:38:47.482612 22553 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:38705
I20260809 05:38:47.482756 22753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:38705 every 8 connection(s)
I20260809 05:38:47.484102 22553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:38:47.485329 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 22553
I20260809 05:38:47.485548 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:38:47.486140 31271 external_mini_cluster.cc:1428] Setting key 0b199d79cd6fa659c83252bee55a9dd0
Aug 09 05:38:47 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:38:47.506184 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.487232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:38207 (local address 127.30.137.254:34445)
0809 05:38:47.487969 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.487974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.489033 (+ 1059us) server_negotiation.cc:407] Connection header received
0809 05:38:47.491192 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.491196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.491286 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.491377 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:47.492831 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.493625 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.495350 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.495844 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.495941 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.499374 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:47.499379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:47.499382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:47.499417 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:47.502984 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.503759 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.503764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.503767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.503898 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.504371 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.504375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.504377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.504665 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:47.504851 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.505557 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.505788 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":479,"threads_started":1}
I20260809 05:38:47.507267 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.487934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:38207)
0809 05:38:47.488852 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:47.488904 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:47.490947 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:47.491582 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:47.491590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:47.492115 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:38:47.492681 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.492699 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.493763 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.493767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.495222 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.495234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.496035 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.496040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.496238 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.496901 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:38:47.496923 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:38:47.499167 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:38:47.503131 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:47.503141 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:47.503164 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:47.503617 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:47.504017 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:38:47.504022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:38:47.504026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:38:47.504236 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:38:47.504803 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:38:47.504811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:38:47.505169 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 05:38:47.505447 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 05:38:47.505999 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":371,"threads_started":1}
I20260809 05:38:47.508706 22755 heartbeater.cc:347] Connected to a master server at 127.30.137.254:34445
I20260809 05:38:47.509020 22755 heartbeater.cc:464] Registering TS with master...
I20260809 05:38:47.509752 22755 heartbeater.cc:511] Master 127.30.137.254:34445 requested a full tablet report, sending...
I20260809 05:38:47.511246 22081 ts_manager.cc:194] Registered new tserver with Master: 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705)
I20260809 05:38:47.512238 22081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:38207
I20260809 05:38:47.514883 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:38:47.532541 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.520025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.30.137.254:34445)
0809 05:38:47.520210 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.520215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.520721 (+ 506us) server_negotiation.cc:407] Connection header received
0809 05:38:47.520889 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.520893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.520965 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.521048 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:47.522436 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.523185 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.524571 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.524976 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.525070 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.527527 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:47.527532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:47.527535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:47.527564 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:47.530205 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.530925 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.530929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.530931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.531046 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.531398 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.531401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.531402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.531709 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:47.531901 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.532155 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.532329 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 05:38:47.536630 22081 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:38:47.543674 22081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:38:47.567886 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.559422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35944 (local address 127.30.137.193:33581)
0809 05:38:47.560133 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.560140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.560162 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:38:47.560258 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.560264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.560486 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.560700 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:47.561614 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.562784 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.565336 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.566628 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.566881 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.567070 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.567351 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.567522 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":375,"threads_started":1}
I20260809 05:38:47.568428 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.559055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:35944)
0809 05:38:47.559612 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:47.559626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:47.559768 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:47.560748 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:47.560757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:47.560805 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:47.561424 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.561441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.562918 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.562926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.565170 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.565184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.566858 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.566865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.567068 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.567132 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 05:38:47.567311 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 05:38:47.567627 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":75,"thread_start_us":298,"threads_started":1}
I20260809 05:38:47.572371 22777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.556361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46597 (local address 127.0.0.1:47742)
0809 05:38:47.557497 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:47.557558 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:47.557793 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:47.563935 (+ 6142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:47.563940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:47.563969 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:47.564953 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.564967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.566986 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.566992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.569430 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.569446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.571446 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.571453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.571635 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.571692 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 05:38:47.571809 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 05:38:47.571947 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":29,"thread_start_us":1815,"threads_started":1}
I20260809 05:38:47.572397 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.556333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.30.137.194:46597)
0809 05:38:47.562950 (+ 6617us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.562964 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.563005 (+ 41us) server_negotiation.cc:407] Connection header received
0809 05:38:47.563136 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.563147 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.563608 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.563912 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:47.565255 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.566814 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.569640 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.571184 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.571425 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.571607 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.571869 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.572042 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":332,"threads_started":1}
I20260809 05:38:47.573807 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.556432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:38705 (local address 127.0.0.1:41790)
0809 05:38:47.558536 (+ 2104us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:47.558559 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:47.558726 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:47.565412 (+ 6686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:47.565419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:47.565448 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:47.565878 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.565892 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.567882 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.567888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.570060 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.570075 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.572775 (+ 2700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.572785 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.572952 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.572983 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:38:47.573052 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 05:38:47.573186 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1938,"mutex_wait_us":1,"thread_start_us":1755,"threads_started":1}
I20260809 05:38:47.574347 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.557033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.30.137.195:38705)
0809 05:38:47.564805 (+ 7772us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.564815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.564856 (+ 41us) server_negotiation.cc:407] Connection header received
0809 05:38:47.564959 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.564966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.565221 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.565434 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:47.566073 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.567693 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.570874 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.572210 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.573539 (+ 1329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.573725 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.573794 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.573961 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7551,"thread_start_us":7393,"threads_started":1}
I20260809 05:38:47.574563 22255 tablet_service.cc:1467] Processing CreateTablet for tablet 42db50ce19bc41499f849e6615779304 (DEFAULT_TABLE table=test-table [id=7d72d5d7e8604f0795ae31445c65c056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:47.576085 22443 tablet_service.cc:1467] Processing CreateTablet for tablet 42db50ce19bc41499f849e6615779304 (DEFAULT_TABLE table=test-table [id=7d72d5d7e8604f0795ae31445c65c056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:47.576699 22255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42db50ce19bc41499f849e6615779304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:47.577263 22443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42db50ce19bc41499f849e6615779304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:47.578080 22655 tablet_service.cc:1467] Processing CreateTablet for tablet 42db50ce19bc41499f849e6615779304 (DEFAULT_TABLE table=test-table [id=7d72d5d7e8604f0795ae31445c65c056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:38:47.579612 22655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42db50ce19bc41499f849e6615779304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:38:47.589094 22781 tablet_bootstrap.cc:492] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Bootstrap starting.
I20260809 05:38:47.592449 22781 tablet_bootstrap.cc:654] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:47.592913 22784 tablet_bootstrap.cc:492] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Bootstrap starting.
I20260809 05:38:47.594100 22781 log.cc:824] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:47.594445 22785 tablet_bootstrap.cc:492] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Bootstrap starting.
I20260809 05:38:47.596848 22784 tablet_bootstrap.cc:654] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:47.598068 22781 tablet_bootstrap.cc:492] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: No bootstrap required, opened a new log
I20260809 05:38:47.598150 22784 log.cc:824] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:47.598380 22781 ts_tablet_manager.cc:1397] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260809 05:38:47.598600 22785 tablet_bootstrap.cc:654] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Neither blocks nor log segments found. Creating new log.
I20260809 05:38:47.599840 22785 log.cc:824] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Log is configured to *not* fsync() on all Append() calls
I20260809 05:38:47.601339 22784 tablet_bootstrap.cc:492] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: No bootstrap required, opened a new log
I20260809 05:38:47.601606 22784 ts_tablet_manager.cc:1397] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 05:38:47.602447 22785 tablet_bootstrap.cc:492] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: No bootstrap required, opened a new log
I20260809 05:38:47.602761 22785 ts_tablet_manager.cc:1397] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260809 05:38:47.609169 22781 raft_consensus.cc:348] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.609664 22781 raft_consensus.cc:374] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:47.609781 22781 raft_consensus.cc:729] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3e2f1a5a324cb985896d007d0e6a32, State: Initialized, Role: FOLLOWER
I20260809 05:38:47.610576 22781 consensus_queue.cc:260] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.610634 22785 raft_consensus.cc:348] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.611065 22785 raft_consensus.cc:374] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:47.611183 22785 raft_consensus.cc:729] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bc2c169dba426b809034133794a8e5, State: Initialized, Role: FOLLOWER
I20260809 05:38:47.611948 22543 heartbeater.cc:503] Master 127.30.137.254:34445 was elected leader, sending a full tablet report...
I20260809 05:38:47.611934 22785 consensus_queue.cc:260] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.612385 22784 raft_consensus.cc:348] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.612746 22781 ts_tablet_manager.cc:1428] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260809 05:38:47.612846 22784 raft_consensus.cc:374] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:38:47.612932 22784 raft_consensus.cc:729] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1457f307f3045c4ad8ea72d8ae584f8, State: Initialized, Role: FOLLOWER
I20260809 05:38:47.613683 22755 heartbeater.cc:503] Master 127.30.137.254:34445 was elected leader, sending a full tablet report...
I20260809 05:38:47.613600 22784 consensus_queue.cc:260] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.614372 22785 ts_tablet_manager.cc:1428] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Time spent starting tablet: real 0.011s user 0.002s sys 0.008s
I20260809 05:38:47.616497 22784 ts_tablet_manager.cc:1428] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Time spent starting tablet: real 0.015s user 0.010s sys 0.007s
W20260809 05:38:47.670331 22545 tablet.cc:2369] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:38:47.675972 22793 raft_consensus.cc:482] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:38:47.676321 22793 raft_consensus.cc:504] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.678231 22793 leader_election.cc:288] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194:46597), 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705)
I20260809 05:38:47.685572 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.678613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48275 (local address 127.30.137.194:46597)
0809 05:38:47.679519 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.679526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.679550 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:38:47.679785 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.679792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.679867 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.679977 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:47.680697 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.681651 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.683725 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.684513 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.684635 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.684755 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.685144 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.685286 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756}
I20260809 05:38:47.685597 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.679013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:40377 (local address 127.30.137.195:38705)
0809 05:38:47.679238 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.679248 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.679793 (+ 545us) server_negotiation.cc:407] Connection header received
0809 05:38:47.679994 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.680001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.680078 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.680185 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:38:47.680901 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.681823 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.683777 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.684549 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.684756 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.684845 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.685203 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.685333 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 05:38:47.685711 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.678533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46597 (local address 127.30.137.193:48275)
0809 05:38:47.679409 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:47.679440 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:47.679607 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:47.680009 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:47.680013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:47.680038 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:47.680499 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.680516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.681829 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.681834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.683557 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.683569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.684769 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.684775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.684975 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.685004 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 05:38:47.685082 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 05:38:47.685235 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":101,"thread_start_us":431,"threads_started":1}
I20260809 05:38:47.685978 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.678733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:38705 (local address 127.30.137.193:40377)
0809 05:38:47.679654 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 05:38:47.679691 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 05:38:47.679842 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:38:47.680215 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:38:47.680217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:38:47.680238 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:38:47.680685 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.680697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.682007 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.682012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.683581 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:38:47.683593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.684842 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.684847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:38:47.685013 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.685036 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:38:47.685152 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 05:38:47.685284 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":41,"thread_start_us":561,"threads_started":1}
I20260809 05:38:47.686879 22477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42db50ce19bc41499f849e6615779304" candidate_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" is_pre_election: true
I20260809 05:38:47.687083 22686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42db50ce19bc41499f849e6615779304" candidate_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bc2c169dba426b809034133794a8e5" is_pre_election: true
I20260809 05:38:47.687561 22477 raft_consensus.cc:2454] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1457f307f3045c4ad8ea72d8ae584f8 in term 0.
I20260809 05:38:47.687608 22686 raft_consensus.cc:2454] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1457f307f3045c4ad8ea72d8ae584f8 in term 0.
I20260809 05:38:47.688390 22192 leader_election.cc:302] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18bc2c169dba426b809034133794a8e5, 7c3e2f1a5a324cb985896d007d0e6a32, a1457f307f3045c4ad8ea72d8ae584f8; no voters:
I20260809 05:38:47.688879 22793 raft_consensus.cc:2781] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:38:47.689018 22793 raft_consensus.cc:482] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:38:47.689172 22793 raft_consensus.cc:3037] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:47.691563 22793 raft_consensus.cc:504] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.692211 22793 leader_election.cc:288] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [CANDIDATE]: Term 1 election: Requested vote from peers 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194:46597), 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705)
I20260809 05:38:47.692768 22686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42db50ce19bc41499f849e6615779304" candidate_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bc2c169dba426b809034133794a8e5"
I20260809 05:38:47.692772 22477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42db50ce19bc41499f849e6615779304" candidate_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c3e2f1a5a324cb985896d007d0e6a32"
I20260809 05:38:47.693005 22477 raft_consensus.cc:3037] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:47.692989 22686 raft_consensus.cc:3037] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:38:47.695415 22686 raft_consensus.cc:2454] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1457f307f3045c4ad8ea72d8ae584f8 in term 1.
I20260809 05:38:47.695605 22477 raft_consensus.cc:2454] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1457f307f3045c4ad8ea72d8ae584f8 in term 1.
I20260809 05:38:47.695940 22194 leader_election.cc:302] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18bc2c169dba426b809034133794a8e5, a1457f307f3045c4ad8ea72d8ae584f8; no voters:
I20260809 05:38:47.696273 22793 raft_consensus.cc:2781] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:38:47.696756 22793 raft_consensus.cc:686] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 LEADER]: Becoming Leader. State: Replica: a1457f307f3045c4ad8ea72d8ae584f8, State: Running, Role: LEADER
I20260809 05:38:47.697242 22793 consensus_queue.cc:237] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:38:47.702241 22081 catalog_manager.cc:5495] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } health_report { overall_health: HEALTHY } } }
I20260809 05:38:47.725159 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.719319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.30.137.193:33581)
0809 05:38:47.719542 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.719549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.719674 (+ 125us) server_negotiation.cc:407] Connection header received
0809 05:38:47.719944 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.719948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.720072 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.720233 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:38:47.720633 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.721349 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.722925 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.723354 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.723499 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.724048 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:38:47.724575 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:38:47.724742 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.724803 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.724887 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 05:38:47.734462 22477 raft_consensus.cc:1261] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Refusing update from remote peer a1457f307f3045c4ad8ea72d8ae584f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:47.734462 22686 raft_consensus.cc:1261] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Refusing update from remote peer a1457f307f3045c4ad8ea72d8ae584f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:38:47.735694 22799 consensus_queue.cc:1035] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:38:47.735872 22757 tablet.cc:2369] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:38:47.736068 22793 consensus_queue.cc:1035] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:38:47.748735 22806 mvcc.cc:205] Tried to move back new op lower bound from 7316496087988363264 to 7316496087851560960. Current Snapshot: MvccSnapshot[applied={T|T < 7316496087988363264}]
I20260809 05:38:47.763860 22081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:47.764149 22081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:38:47.767673 22081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Sending DeleteTablet for 3 replicas of tablet 42db50ce19bc41499f849e6615779304
I20260809 05:38:47.768918 22443 tablet_service.cc:1514] Processing DeleteTablet for tablet 42db50ce19bc41499f849e6615779304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:47 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:47742
I20260809 05:38:47.769061 22655 tablet_service.cc:1514] Processing DeleteTablet for tablet 42db50ce19bc41499f849e6615779304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:47 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:41790
I20260809 05:38:47.769181 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 22165
I20260809 05:38:47.770220 22816 tablet_replica.cc:330] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: stopping tablet replica
I20260809 05:38:47.770720 22816 raft_consensus.cc:2229] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:47.771262 22816 raft_consensus.cc:2258] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:47.770641 22255 tablet_service.cc:1514] Processing DeleteTablet for tablet 42db50ce19bc41499f849e6615779304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:38:47 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:35944
I20260809 05:38:47.775254 22817 tablet_replica.cc:330] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: stopping tablet replica
I20260809 05:38:47.775483 22819 tablet_replica.cc:330] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: stopping tablet replica
I20260809 05:38:47.775695 22817 raft_consensus.cc:2229] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:38:47.775898 22819 raft_consensus.cc:2229] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:38:47.776218 22817 raft_consensus.cc:2258] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:47.776654 22816 ts_tablet_manager.cc:1905] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:47.776609 22819 raft_consensus.cc:2258] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:38:47.778373 22817 ts_tablet_manager.cc:1905] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:47.779876 22819 ts_tablet_manager.cc:1905] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:38:47.784392 22816 ts_tablet_manager.cc:1918] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:47.784631 22816 log.cc:1196] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/42db50ce19bc41499f849e6615779304
I20260809 05:38:47.785275 22816 ts_tablet_manager.cc:1939] T 42db50ce19bc41499f849e6615779304 P 7c3e2f1a5a324cb985896d007d0e6a32: Deleting consensus metadata
I20260809 05:38:47.785801 22817 ts_tablet_manager.cc:1918] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:47.785971 22817 log.cc:1196] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/42db50ce19bc41499f849e6615779304
I20260809 05:38:47.786547 22817 ts_tablet_manager.cc:1939] T 42db50ce19bc41499f849e6615779304 P 18bc2c169dba426b809034133794a8e5: Deleting consensus metadata
I20260809 05:38:47.787617 22057 catalog_manager.cc:4841] TS 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194:46597): tablet 42db50ce19bc41499f849e6615779304 (table test-table [id=7d72d5d7e8604f0795ae31445c65c056]) successfully deleted
I20260809 05:38:47.788350 22061 catalog_manager.cc:4841] TS 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705): tablet 42db50ce19bc41499f849e6615779304 (table test-table [id=7d72d5d7e8604f0795ae31445c65c056]) successfully deleted
I20260809 05:38:47.790519 22819 ts_tablet_manager.cc:1918] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:38:47.790740 22819 log.cc:1196] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/42db50ce19bc41499f849e6615779304
I20260809 05:38:47.791226 22819 ts_tablet_manager.cc:1939] T 42db50ce19bc41499f849e6615779304 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting consensus metadata
Aug 09 05:38:47 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:38:47.793143 22061 catalog_manager.cc:4841] TS a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193:33581): tablet 42db50ce19bc41499f849e6615779304 (table test-table [id=7d72d5d7e8604f0795ae31445c65c056]) successfully deleted
I20260809 05:38:47.801102 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.771003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.30.137.193:33581)
0809 05:38:47.771177 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.771182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.771201 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:38:47.771258 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.771263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.771340 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.771467 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:47.777045 (+ 5578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.778173 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.784841 (+ 6668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.785654 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.785770 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.793559 (+ 7789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:47.793574 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:47.793592 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:47.793636 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:47.797943 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.798748 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.798762 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.798766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.798951 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.799362 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.799367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.799371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.799837 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:47.800194 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.800442 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.800710 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 05:38:47.801899 22344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:47.903733 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 22165
I20260809 05:38:47.930902 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 22361
Aug 09 05:38:47 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:38:47.947151 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:47.931792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.30.137.194:46597)
0809 05:38:47.932020 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:47.932026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:47.932048 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:38:47.932157 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:47.932163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:47.932247 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:47.932359 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:47.933751 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.934720 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.936132 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:47.936542 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:47.936685 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:47.941081 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:47.941094 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:47.941108 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:47.941141 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:47.944099 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.944870 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.944878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.944880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.945008 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:47.945407 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:47.945412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:47.945415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:47.945835 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:47.946167 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:47.946509 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 05:38:47.946770 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 05:38:47.947862 22536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:48.038951 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 22361
I20260809 05:38:48.063907 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 22553
Aug 09 05:38:48 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:38:48.079686 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:38:48.064771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.30.137.195:38705)
0809 05:38:48.064948 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 05:38:48.064952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:38:48.064973 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:38:48.065038 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:38:48.065041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:38:48.065103 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:38:48.065202 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:38:48.066589 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:48.067513 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:48.068845 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:38:48.069292 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:38:48.069404 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:38:48.074220 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:38:48.074239 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:38:48.074255 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:38:48.074292 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:38:48.077125 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:48.077828 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:48.077836 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:48.077838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:48.077963 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:38:48.078293 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:38:48.078296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:38:48.078298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:38:48.078728 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:38:48.078985 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 05:38:48.079258 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 05:38:48.079430 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:38:48.080412 22748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:48.176206 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 22553
I20260809 05:38:48.199098 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 22031
I20260809 05:38:48.200417 22120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:38:48.312947 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 22031
I20260809 05:39:08.345502 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34445
--webserver_interface=127.30.137.254
--webserver_port=37405
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:34445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:08.703243 22825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:08.703563 22825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:08.703734 22825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:08.711006 22825 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 05:39:08.711109 22825 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 05:39:08.711193 22825 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 05:39:08.711302 22825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:39:08.711405 22825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:08.711536 22825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:08.711616 22825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:39:08.711719 22825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:39:08.721675 22825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:34445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=37405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:08.723047 22825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:08.724421 22825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:08.751344 22832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:08.751323 22835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:08.751464 22833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:08.752290 22825 server_base.cc:1034] running on GCE node
I20260809 05:39:08.753278 22825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:08.754990 22825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:08.756251 22825 hybrid_clock.cc:648] HybridClock initialized: now 1786253948756173 us; error 68 us; skew 500 ppm
I20260809 05:39:08.756664 22825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:08 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253948, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:08.763998 22825 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:39:08.765631 22825 webserver.cc:460] Webserver started at http://127.30.137.254:37405/ using document root <none> and password file <none>
I20260809 05:39:08.766458 22825 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:08.766618 22825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:08.773461 22825 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 05:39:08.776731 22846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:08.778178 22825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 05:39:08.778378 22825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "a6ce0cada05b43fe9b6b37ee413c6585"
format_stamp: "Formatted at 2026-08-09 05:38:45 on dist-test-slave-3kk6"
server_key: "f97fdffc6013344f99b8f612d9b73c6e"
server_key_iv: "0979adf0b3918a894c0b3e6f4da1aa90"
server_key_version: "encryptionkey@0"
I20260809 05:39:08.779537 22825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:08.795704 22825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:08.800382 22825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:08.800731 22825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:08.819890 22825 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:34445
I20260809 05:39:08.819921 22921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:34445 every 8 connection(s)
I20260809 05:39:08.821573 22825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:39:08.828974 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 22825
I20260809 05:39:08.829097 22922 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:39:08.830096 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:33581
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:34445
--webserver_port=35461
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:39:08.834482 22922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap starting.
I20260809 05:39:08.857856 22922 log.cc:824] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:08.870442 22922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:39:08.871194 22922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap complete.
I20260809 05:39:08.880926 22922 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:08.881762 22922 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ce0cada05b43fe9b6b37ee413c6585, State: Initialized, Role: FOLLOWER
I20260809 05:39:08.882473 22922 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:08.882776 22922 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:39:08.882918 22922 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:39:08.883118 22922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 1 FOLLOWER]: Advancing to term 2
I20260809 05:39:08.887480 22922 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:08.888026 22922 leader_election.cc:302] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6ce0cada05b43fe9b6b37ee413c6585; no voters:
I20260809 05:39:08.888742 22922 leader_election.cc:288] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 05:39:08.888900 22928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 05:39:08.891479 22922 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:39:08.896059 22928 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 LEADER]: Becoming Leader. State: Replica: a6ce0cada05b43fe9b6b37ee413c6585, State: Running, Role: LEADER
I20260809 05:39:08.896910 22928 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:08.899978 22929 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } } }
I20260809 05:39:08.900425 22929 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:08.901054 22930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6ce0cada05b43fe9b6b37ee413c6585. Latest consensus state: current_term: 2 leader_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } } }
I20260809 05:39:08.901364 22930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:08.907313 22950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:39:08.913818 22950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:39:08.914952 22950 catalog_manager.cc:1259] Loaded cluster ID: fa4302127733423d95194e451793f5db
I20260809 05:39:08.915064 22950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:39:08.918254 22950 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:39:08.919482 22950 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Loaded TSK: 0
I20260809 05:39:08.940642 22950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Generated new TSK 1
I20260809 05:39:08.941046 22950 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 05:39:09.212082 22924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:09.212435 22924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:09.212607 22924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:09.220294 22924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:09.220469 22924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:09.220649 22924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:39:09.230791 22924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:33581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=35461
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:09.232200 22924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:09.233601 22924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:09.248400 22963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:09.248857 22964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:09.248899 22966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:09.250731 22924 server_base.cc:1034] running on GCE node
I20260809 05:39:09.251411 22924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:09.252453 22924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:09.253718 22924 hybrid_clock.cc:648] HybridClock initialized: now 1786253949253644 us; error 46 us; skew 500 ppm
I20260809 05:39:09.254099 22924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:09 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253949, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:09.257870 22924 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:39:09.259774 22924 webserver.cc:460] Webserver started at http://127.30.137.193:35461/ using document root <none> and password file <none>
I20260809 05:39:09.260694 22924 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:09.260859 22924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:09.268404 22924 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 05:39:09.272100 22980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:09.273753 22924 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 05:39:09.273947 22924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "a1457f307f3045c4ad8ea72d8ae584f8"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "6c51c439ac1d289e9c6237d1e8f02b6f"
server_key_iv: "598a54630d63ae9204712e080ca4a0f3"
server_key_version: "encryptionkey@0"
I20260809 05:39:09.275094 22924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:09.288825 22924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:09.293727 22924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:09.294025 22924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:09.295222 22924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:09.296655 22924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:09.296788 22924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:09.296912 22924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:09.296998 22924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:09.332165 22924 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:33581
I20260809 05:39:09.332223 23159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:33581 every 8 connection(s)
I20260809 05:39:09.333748 22924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:39:09.335088 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 22924
I20260809 05:39:09.335970 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:46597
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:34445
--webserver_port=42741
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:39:09 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253949, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:09.364547 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:09.337677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:48415)
0809 05:39:09.338710 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:09.338777 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:09.341646 (+ 2869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:09.343445 (+ 1799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:09.343459 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:09.344355 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:09.345200 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:09.345230 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.347001 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.347010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:09.349267 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:09.349289 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.351630 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.351635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:09.351858 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:09.352652 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:39:09.352677 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:09.355048 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:09.360158 (+ 5110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:09.360166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:09.360179 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:09.360632 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:09.361162 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:09.361166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:09.361168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:09.361358 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:09.362416 (+ 1058us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:09.362422 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:09.362806 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 05:39:09.363064 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 05:39:09.363487 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":412,"threads_started":1}
I20260809 05:39:09.365134 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:09.337516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:48415 (local address 127.30.137.254:34445)
0809 05:39:09.338300 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:09.338319 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:09.338955 (+ 636us) server_negotiation.cc:407] Connection header received
0809 05:39:09.342345 (+ 3390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:09.342360 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:09.342874 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:09.343212 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:09.345441 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.346806 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.349452 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.351386 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.351622 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:09.355386 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:09.355399 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:09.355422 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:09.355476 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:09.359984 (+ 4508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:09.360792 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:09.360804 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:09.360814 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:09.361036 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:09.361476 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:09.361483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:09.361489 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:09.362270 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:09.362473 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:09.363334 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 05:39:09.363840 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":294,"threads_started":1}
I20260809 05:39:09.368090 23160 heartbeater.cc:347] Connected to a master server at 127.30.137.254:34445
I20260809 05:39:09.368463 23160 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:09.369472 23160 heartbeater.cc:511] Master 127.30.137.254:34445 requested a full tablet report, sending...
I20260809 05:39:09.372023 22869 ts_manager.cc:194] Registered new tserver with Master: a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193:33581)
I20260809 05:39:09.373500 22869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:48415
W20260809 05:39:09.697494 23165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:09.697810 23165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:09.697975 23165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:09.706292 23165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:09.706424 23165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:09.706636 23165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:39:09.716905 23165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:46597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=42741
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:09.718380 23165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:09.719807 23165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:09.733826 23172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:09.733901 23175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:09.733924 23173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:09.735065 23165 server_base.cc:1034] running on GCE node
I20260809 05:39:09.735733 23165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:09.736635 23165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:09.737874 23165 hybrid_clock.cc:648] HybridClock initialized: now 1786253949737818 us; error 32 us; skew 500 ppm
I20260809 05:39:09.738175 23165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:09 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253949, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:09.741539 23165 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:39:09.743184 23165 webserver.cc:460] Webserver started at http://127.30.137.194:42741/ using document root <none> and password file <none>
I20260809 05:39:09.743907 23165 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:09.744076 23165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:09.749996 23165 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 05:39:09.753024 23183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:09.754464 23165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:39:09.754736 23165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "7c3e2f1a5a324cb985896d007d0e6a32"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "1b362ec57a26b3abca7462012c3a2210"
server_key_iv: "5adc0fb1815ad435fe0c44e17b91ecc4"
server_key_version: "encryptionkey@0"
I20260809 05:39:09.755820 23165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:09.781816 23165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:09.786235 23165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:09.786532 23165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:09.787618 23165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:09.789006 23165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:09.789141 23165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:09.789271 23165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:09.789361 23165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:09.826426 23165 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46597
I20260809 05:39:09.826483 23356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46597 every 8 connection(s)
I20260809 05:39:09.827976 23165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:39:09.834642 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 23165
I20260809 05:39:09.835522 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:38705
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:34445
--webserver_port=40685
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 05:39:09 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253949, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:09.853016 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:09.830874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:59507 (local address 127.30.137.254:34445)
0809 05:39:09.831096 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:09.831103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:09.832430 (+ 1327us) server_negotiation.cc:407] Connection header received
0809 05:39:09.835202 (+ 2772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:09.835208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:09.835334 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:09.835493 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:09.837645 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.838824 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.840456 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.841224 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.841378 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:09.845318 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:09.845327 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:09.845334 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:09.845383 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:09.848905 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:09.849907 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:09.849911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:09.849913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:09.850035 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:09.850508 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:09.850516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:09.850523 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:09.851012 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:09.851168 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:09.852260 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0809 05:39:09.852542 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:39:09.853806 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:09.831388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:59507)
0809 05:39:09.832227 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:09.832276 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:09.834851 (+ 2575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:09.835870 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:09.835881 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:09.836654 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:09.837417 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:09.837444 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.839000 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.839005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:09.840293 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:09.840307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:09.841451 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:09.841456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:09.841657 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:09.842451 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:39:09.842474 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:09.845039 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:09.849064 (+ 4025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:09.849076 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:09.849095 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:09.849765 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:09.850138 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:09.850142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:09.850144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:09.850375 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:09.851215 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:09.851225 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:09.851792 (+ 567us) client_negotiation.cc:769] Sending connection context
0809 05:39:09.852138 (+ 346us) client_negotiation.cc:240] Negotiation successful
0809 05:39:09.852562 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":335,"threads_started":1}
I20260809 05:39:09.855541 23357 heartbeater.cc:347] Connected to a master server at 127.30.137.254:34445
I20260809 05:39:09.855911 23357 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:09.856918 23357 heartbeater.cc:511] Master 127.30.137.254:34445 requested a full tablet report, sending...
I20260809 05:39:09.858724 22869 ts_manager.cc:194] Registered new tserver with Master: 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194:46597)
I20260809 05:39:09.859669 22869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:59507
W20260809 05:39:10.207894 23361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:10.208220 23361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:10.208321 23361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:10.215828 23361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:10.215957 23361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:10.216135 23361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:39:10.226191 23361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=40685
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:10.227579 23361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:10.228909 23361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:10.242587 23368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:10.242563 23374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:10.242759 23370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:10.244755 23361 server_base.cc:1034] running on GCE node
I20260809 05:39:10.245488 23361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:10.246518 23361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:10.247776 23361 hybrid_clock.cc:648] HybridClock initialized: now 1786253950247730 us; error 61 us; skew 500 ppm
I20260809 05:39:10.248127 23361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:10 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253950, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:10.251488 23361 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:39:10.253087 23361 webserver.cc:460] Webserver started at http://127.30.137.195:40685/ using document root <none> and password file <none>
I20260809 05:39:10.253852 23361 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:10.254040 23361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:10.259984 23361 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 05:39:10.263099 23389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:10.264705 23361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 05:39:10.264937 23361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "18bc2c169dba426b809034133794a8e5"
format_stamp: "Formatted at 2026-08-09 05:38:47 on dist-test-slave-3kk6"
server_key: "2133b753e7458c73e2187894cf70b7fa"
server_key_iv: "8d632ed6c3e187a3624e0c86d96a4fa6"
server_key_version: "encryptionkey@0"
I20260809 05:39:10.266121 23361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:10.278203 23361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:10.282860 23361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:10.283175 23361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:10.284358 23361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:10.285771 23361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:10.285907 23361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:10.286055 23361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:10.286155 23361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:10.322245 23361 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:38705
I20260809 05:39:10.322304 23560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:38705 every 8 connection(s)
I20260809 05:39:10.323947 23361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:39:10.333595 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 23361
Aug 09 05:39:10 dist-test-slave-3kk6 krb5kdc[22012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253950, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:10.348438 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.326620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:47631 (local address 127.30.137.254:34445)
0809 05:39:10.326846 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.326851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.328067 (+ 1216us) server_negotiation.cc:407] Connection header received
0809 05:39:10.330214 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.330220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.330316 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.330435 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:10.332141 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.332885 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.334826 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.335791 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.335932 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.341253 (+ 5321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:10.341258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:10.341261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:10.341297 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:10.344747 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.345719 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.345735 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.345741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.345902 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.346357 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.346364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.346368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.346819 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:10.347001 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.347837 (+ 836us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.348077 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 05:39:10.349844 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.327156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:47631)
0809 05:39:10.327889 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.327929 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:10.329914 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:10.330759 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:10.330768 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:10.331351 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:10.331954 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.331974 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.333062 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.333066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.334614 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.334633 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.336529 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.336539 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.336953 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.337890 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:39:10.337921 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:10.340990 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:10.344913 (+ 3923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:10.344926 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:10.344953 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:10.345581 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:10.346017 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:10.346021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:10.346023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:10.346221 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:10.346949 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:10.346955 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:10.347355 (+ 400us) client_negotiation.cc:769] Sending connection context
0809 05:39:10.347686 (+ 331us) client_negotiation.cc:240] Negotiation successful
0809 05:39:10.348271 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":291,"threads_started":1}
I20260809 05:39:10.351588 23561 heartbeater.cc:347] Connected to a master server at 127.30.137.254:34445
I20260809 05:39:10.351938 23561 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:10.352938 23561 heartbeater.cc:511] Master 127.30.137.254:34445 requested a full tablet report, sending...
I20260809 05:39:10.354331 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.334770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.30.137.254:34445)
0809 05:39:10.338430 (+ 3660us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.338437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.338595 (+ 158us) server_negotiation.cc:407] Connection header received
0809 05:39:10.338842 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.338848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.338928 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.339045 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:10.341044 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.341994 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.344170 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.344636 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.344738 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.348186 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:10.348195 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:10.348201 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:10.348237 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:10.351389 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.352231 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.352237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.352241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.352393 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.352801 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.352806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.352809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.353194 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:10.353489 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.353708 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.353934 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3509,"thread_start_us":3371,"threads_started":1}
I20260809 05:39:10.354810 22869 ts_manager.cc:194] Registered new tserver with Master: 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705)
I20260809 05:39:10.355629 22869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:47631
I20260809 05:39:10.357168 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:39:10.364024 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.358156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46784 (local address 127.30.137.254:34445)
0809 05:39:10.358308 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.358312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.358938 (+ 626us) server_negotiation.cc:407] Connection header received
0809 05:39:10.359199 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.359203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.359368 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.359539 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:39:10.359984 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.360707 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.362154 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.362585 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.362752 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.363171 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:39:10.363491 (+ 320us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 05:39:10.363780 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46784: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 05:39:10.364173 23567 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46784: token expired
W20260809 05:39:10.364705 22762 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.137.254:34445)
I20260809 05:39:10.377229 23160 heartbeater.cc:503] Master 127.30.137.254:34445 was elected leader, sending a full tablet report...
I20260809 05:39:10.378206 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.365327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.30.137.254:34445)
0809 05:39:10.365483 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.365487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.365519 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:39:10.365555 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.365559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.365609 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.365679 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:10.367184 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.367917 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.369423 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.369895 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.369990 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.372864 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:10.372869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:10.372873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:10.372907 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:10.375467 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.376135 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.376139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.376141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.376280 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.376602 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.376605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.376607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.376934 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:10.377125 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.377539 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.377782 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 05:39:10.381950 22869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:39:10.389606 22869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:39:10.425603 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.403293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.30.137.194:46597)
0809 05:39:10.415181 (+ 11888us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.415194 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.416351 (+ 1157us) server_negotiation.cc:407] Connection header received
0809 05:39:10.416498 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.416510 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.416862 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.417109 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:10.418529 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.419983 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.422468 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.424076 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.424263 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.424491 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.425045 (+ 554us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.425241 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11644,"thread_start_us":11914,"threads_started":1}
I20260809 05:39:10.425637 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.402552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46597 (local address 127.0.0.1:54994)
0809 05:39:10.415977 (+ 13425us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.416005 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:10.416181 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:10.417533 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:10.417547 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:10.417594 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:10.418222 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.418240 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.420155 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.420164 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.422267 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.422284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.424531 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.424537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.424771 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.424824 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 05:39:10.424977 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 05:39:10.425175 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13243,"mutex_wait_us":2,"thread_start_us":13082,"threads_started":1}
I20260809 05:39:10.426635 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.402193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:38705 (local address 127.0.0.1:34872)
0809 05:39:10.409637 (+ 7444us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.409678 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:10.409892 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:10.418370 (+ 8478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:10.418377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:10.418410 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:10.418825 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.418849 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.421614 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.421619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.423817 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.423830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.425783 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.425793 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.425981 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.426018 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:39:10.426084 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 05:39:10.426214 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7160,"mutex_wait_us":118,"thread_start_us":6994,"threads_started":1}
I20260809 05:39:10.427235 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.415453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54670)
0809 05:39:10.418057 (+ 2604us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.418078 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:10.418218 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:10.419798 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:10.419803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:10.419832 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:10.420323 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.420347 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.422104 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.422110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.424325 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.424340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.426449 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.426454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.426573 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.426599 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 05:39:10.426747 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 05:39:10.426901 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2363,"thread_start_us":2183,"threads_started":1}
I20260809 05:39:10.427527 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.410520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.30.137.193:33581)
0809 05:39:10.418899 (+ 8379us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.418910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.418940 (+ 30us) server_negotiation.cc:407] Connection header received
0809 05:39:10.419047 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.419054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.419290 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.419522 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:10.420560 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.421958 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.424523 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.425920 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.426112 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.426301 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.426983 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.427158 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8170,"thread_start_us":7968,"threads_started":1}
I20260809 05:39:10.427841 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.402633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.30.137.195:38705)
0809 05:39:10.417590 (+ 14957us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.417601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.417648 (+ 47us) server_negotiation.cc:407] Connection header received
0809 05:39:10.417759 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.417768 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.418128 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.419057 (+ 929us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:10.419401 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.421439 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.424014 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.425511 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.427046 (+ 1535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.427220 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.427305 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.427478 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14756,"thread_start_us":14549,"threads_started":1}
I20260809 05:39:10.430243 23257 tablet_service.cc:1467] Processing CreateTablet for tablet dff7f1236b9643b9be0242601ff45f99 (DEFAULT_TABLE table=test-table [id=6f664d81234c44b480ecd1483e2cd3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:10.431569 23257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff7f1236b9643b9be0242601ff45f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:10.431416 23055 tablet_service.cc:1467] Processing CreateTablet for tablet dff7f1236b9643b9be0242601ff45f99 (DEFAULT_TABLE table=test-table [id=6f664d81234c44b480ecd1483e2cd3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:10.431416 23458 tablet_service.cc:1467] Processing CreateTablet for tablet dff7f1236b9643b9be0242601ff45f99 (DEFAULT_TABLE table=test-table [id=6f664d81234c44b480ecd1483e2cd3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:10.432885 23458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff7f1236b9643b9be0242601ff45f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:10.432915 23055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dff7f1236b9643b9be0242601ff45f99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:10.444707 23583 tablet_bootstrap.cc:492] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Bootstrap starting.
I20260809 05:39:10.448870 23583 tablet_bootstrap.cc:654] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:10.449976 23583 log.cc:824] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:10.451026 23584 tablet_bootstrap.cc:492] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Bootstrap starting.
I20260809 05:39:10.452131 23585 tablet_bootstrap.cc:492] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Bootstrap starting.
I20260809 05:39:10.455055 23583 tablet_bootstrap.cc:492] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: No bootstrap required, opened a new log
I20260809 05:39:10.455363 23583 ts_tablet_manager.cc:1397] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260809 05:39:10.456029 23584 tablet_bootstrap.cc:654] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:10.457350 23584 log.cc:824] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:10.458124 23585 tablet_bootstrap.cc:654] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:10.459465 23585 log.cc:824] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:10.460968 23584 tablet_bootstrap.cc:492] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: No bootstrap required, opened a new log
I20260809 05:39:10.461257 23584 ts_tablet_manager.cc:1397] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 05:39:10.462286 23585 tablet_bootstrap.cc:492] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: No bootstrap required, opened a new log
I20260809 05:39:10.462566 23585 ts_tablet_manager.cc:1397] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260809 05:39:10.466292 23583 raft_consensus.cc:348] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.466796 23583 raft_consensus.cc:374] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:10.466890 23583 raft_consensus.cc:729] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3e2f1a5a324cb985896d007d0e6a32, State: Initialized, Role: FOLLOWER
I20260809 05:39:10.467567 23583 consensus_queue.cc:260] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.468966 23357 heartbeater.cc:503] Master 127.30.137.254:34445 was elected leader, sending a full tablet report...
I20260809 05:39:10.469656 23583 ts_tablet_manager.cc:1428] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 05:39:10.471765 23584 raft_consensus.cc:348] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.471740 23585 raft_consensus.cc:348] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.472158 23584 raft_consensus.cc:374] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:10.472206 23585 raft_consensus.cc:374] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:10.472282 23584 raft_consensus.cc:729] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1457f307f3045c4ad8ea72d8ae584f8, State: Initialized, Role: FOLLOWER
I20260809 05:39:10.472321 23585 raft_consensus.cc:729] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18bc2c169dba426b809034133794a8e5, State: Initialized, Role: FOLLOWER
I20260809 05:39:10.472916 23584 consensus_queue.cc:260] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.473047 23585 consensus_queue.cc:260] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.474807 23584 ts_tablet_manager.cc:1428] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260809 05:39:10.475900 23561 heartbeater.cc:503] Master 127.30.137.254:34445 was elected leader, sending a full tablet report...
I20260809 05:39:10.477276 23585 ts_tablet_manager.cc:1428] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260809 05:39:10.508972 23590 raft_consensus.cc:482] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:39:10.509289 23590 raft_consensus.cc:504] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.510993 23590 leader_election.cc:288] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705), a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193:33581)
I20260809 05:39:10.518708 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.511539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:49635 (local address 127.30.137.195:38705)
0809 05:39:10.511740 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.511744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.512358 (+ 614us) server_negotiation.cc:407] Connection header received
0809 05:39:10.512685 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.512693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.512777 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.512847 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:10.513610 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.514353 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.516421 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.517605 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.517732 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.517787 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.518238 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.518406 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 05:39:10.519080 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.511428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:38705 (local address 127.30.137.194:49635)
0809 05:39:10.512231 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.512252 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:10.512433 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:10.513043 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:10.513046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:10.513068 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:10.513445 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.513454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.514500 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.514507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.516237 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.516255 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.517851 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.517859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.518033 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.518084 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 05:39:10.518182 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 05:39:10.518377 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"spinlock_wait_cycles":19840,"thread_start_us":440,"threads_started":1}
I20260809 05:39:10.519177 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.512529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:60197 (local address 127.30.137.193:33581)
0809 05:39:10.512759 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.512766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.513068 (+ 302us) server_negotiation.cc:407] Connection header received
0809 05:39:10.513288 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.513296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.513361 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.513463 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:10.514003 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.514799 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.516908 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.517988 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.518158 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.518232 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.518750 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.518922 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:39:10.519186 23598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.512319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.30.137.194:60197)
0809 05:39:10.512948 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.512967 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:10.513139 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:10.513484 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:10.513488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:10.513514 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:10.513854 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.513863 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.514938 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.514948 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.516708 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:10.516722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.518318 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.518325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:10.518515 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.518550 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 05:39:10.518699 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 05:39:10.518877 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":311,"threads_started":1}
I20260809 05:39:10.520282 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff7f1236b9643b9be0242601ff45f99" candidate_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" is_pre_election: true
I20260809 05:39:10.520354 23498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff7f1236b9643b9be0242601ff45f99" candidate_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bc2c169dba426b809034133794a8e5" is_pre_election: true
I20260809 05:39:10.520807 23084 raft_consensus.cc:2454] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3e2f1a5a324cb985896d007d0e6a32 in term 0.
I20260809 05:39:10.520829 23498 raft_consensus.cc:2454] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c3e2f1a5a324cb985896d007d0e6a32 in term 0.
I20260809 05:39:10.521482 23188 leader_election.cc:302] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3e2f1a5a324cb985896d007d0e6a32, a1457f307f3045c4ad8ea72d8ae584f8; no voters:
I20260809 05:39:10.521876 23590 raft_consensus.cc:2781] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:39:10.522048 23590 raft_consensus.cc:482] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:39:10.522164 23590 raft_consensus.cc:3037] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:10.524549 23590 raft_consensus.cc:504] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.525141 23590 leader_election.cc:288] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [CANDIDATE]: Term 1 election: Requested vote from peers 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705), a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193:33581)
I20260809 05:39:10.525674 23498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff7f1236b9643b9be0242601ff45f99" candidate_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18bc2c169dba426b809034133794a8e5"
I20260809 05:39:10.525759 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dff7f1236b9643b9be0242601ff45f99" candidate_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1457f307f3045c4ad8ea72d8ae584f8"
I20260809 05:39:10.525913 23498 raft_consensus.cc:3037] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:10.525977 23084 raft_consensus.cc:3037] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:10.528374 23084 raft_consensus.cc:2454] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3e2f1a5a324cb985896d007d0e6a32 in term 1.
I20260809 05:39:10.528465 23498 raft_consensus.cc:2454] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c3e2f1a5a324cb985896d007d0e6a32 in term 1.
I20260809 05:39:10.528844 23188 leader_election.cc:302] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c3e2f1a5a324cb985896d007d0e6a32, a1457f307f3045c4ad8ea72d8ae584f8; no voters:
I20260809 05:39:10.529166 23590 raft_consensus.cc:2781] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:39:10.529662 23590 raft_consensus.cc:686] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 LEADER]: Becoming Leader. State: Replica: 7c3e2f1a5a324cb985896d007d0e6a32, State: Running, Role: LEADER
I20260809 05:39:10.530155 23590 consensus_queue.cc:237] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } }
I20260809 05:39:10.534883 22868 catalog_manager.cc:5495] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194). New cstate: current_term: 1 leader_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c3e2f1a5a324cb985896d007d0e6a32" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 } health_report { overall_health: UNKNOWN } } }
I20260809 05:39:10.554081 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.548635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.30.137.194:46597)
0809 05:39:10.548869 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.548874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.548894 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:39:10.549006 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.549010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.549100 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.549245 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:39:10.549640 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.550416 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.551994 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.552460 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.552563 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.553076 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:39:10.553595 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:39:10.553723 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.553789 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.553864 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 05:39:10.563442 23084 raft_consensus.cc:1261] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Refusing update from remote peer 7c3e2f1a5a324cb985896d007d0e6a32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:39:10.563442 23498 raft_consensus.cc:1261] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Refusing update from remote peer 7c3e2f1a5a324cb985896d007d0e6a32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:39:10.564431 23599 consensus_queue.cc:1035] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1457f307f3045c4ad8ea72d8ae584f8" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 33581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:39:10.564832 23590 consensus_queue.cc:1035] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18bc2c169dba426b809034133794a8e5" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:39:10.575938 23563 tablet.cc:2369] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:39:10.595981 22869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:39:10.596325 22869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:39:10.600459 22869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Sending DeleteTablet for 3 replicas of tablet dff7f1236b9643b9be0242601ff45f99
I20260809 05:39:10.601898 23257 tablet_service.cc:1514] Processing DeleteTablet for tablet dff7f1236b9643b9be0242601ff45f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:10 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54994
I20260809 05:39:10.602239 23458 tablet_service.cc:1514] Processing DeleteTablet for tablet dff7f1236b9643b9be0242601ff45f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:10 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:34872
I20260809 05:39:10.602408 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 22924
I20260809 05:39:10.603138 23055 tablet_service.cc:1514] Processing DeleteTablet for tablet dff7f1236b9643b9be0242601ff45f99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:10 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54670
I20260809 05:39:10.615281 23621 tablet_replica.cc:330] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: stopping tablet replica
I20260809 05:39:10.615973 23621 raft_consensus.cc:2229] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:39:10.616658 23621 raft_consensus.cc:2258] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:10.617136 23619 tablet_replica.cc:330] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: stopping tablet replica
I20260809 05:39:10.618444 23619 raft_consensus.cc:2229] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:39:10.619290 23619 raft_consensus.cc:2258] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:10.619302 23621 ts_tablet_manager.cc:1905] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:10.620045 23620 tablet_replica.cc:330] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: stopping tablet replica
I20260809 05:39:10.620580 23620 raft_consensus.cc:2229] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:39:10.621219 23620 raft_consensus.cc:2258] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:10.621294 23619 ts_tablet_manager.cc:1905] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:10.623364 23620 ts_tablet_manager.cc:1905] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:10.629525 23619 ts_tablet_manager.cc:1918] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:10.629607 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.603790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.30.137.193:33581)
0809 05:39:10.604623 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.604633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.604675 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:39:10.604753 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.604763 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.604875 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.605019 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:10.608176 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.609912 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.612953 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.614081 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.614225 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.619456 (+ 5231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:10.619473 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:10.619492 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:10.619538 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:10.624708 (+ 5170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.625820 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.625830 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.625834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.626029 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.626572 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.626578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.626582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.627140 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:10.627490 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.628881 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.629148 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647}
I20260809 05:39:10.629746 23619 log.cc:1196] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/dff7f1236b9643b9be0242601ff45f99
I20260809 05:39:10.630497 23619 ts_tablet_manager.cc:1939] T dff7f1236b9643b9be0242601ff45f99 P 7c3e2f1a5a324cb985896d007d0e6a32: Deleting consensus metadata
I20260809 05:39:10.630505 23150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:10.632293 22849 catalog_manager.cc:4841] TS 7c3e2f1a5a324cb985896d007d0e6a32 (127.30.137.194:46597): tablet dff7f1236b9643b9be0242601ff45f99 (table test-table [id=6f664d81234c44b480ecd1483e2cd3bc]) successfully deleted
I20260809 05:39:10.637070 23620 ts_tablet_manager.cc:1918] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:10.637282 23620 log.cc:1196] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/dff7f1236b9643b9be0242601ff45f99
I20260809 05:39:10.637895 23620 ts_tablet_manager.cc:1939] T dff7f1236b9643b9be0242601ff45f99 P 18bc2c169dba426b809034133794a8e5: Deleting consensus metadata
I20260809 05:39:10.640617 22851 catalog_manager.cc:4841] TS 18bc2c169dba426b809034133794a8e5 (127.30.137.195:38705): tablet dff7f1236b9643b9be0242601ff45f99 (table test-table [id=6f664d81234c44b480ecd1483e2cd3bc]) successfully deleted
I20260809 05:39:10.759259 23621 ts_tablet_manager.cc:1918] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:10.759582 23621 log.cc:1196] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/dff7f1236b9643b9be0242601ff45f99
I20260809 05:39:10.760258 23621 ts_tablet_manager.cc:1939] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting consensus metadata
I20260809 05:39:10.760251 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 22924
I20260809 05:39:10.787067 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 23165
W20260809 05:39:10.786998 22851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.137.193:33581 (error 108)
I20260809 05:39:10.789160 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.788661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54684)
0809 05:39:10.788841 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.788972 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 05:39:10.790472 22851 catalog_manager.cc:4584] TS a1457f307f3045c4ad8ea72d8ae584f8 (127.30.137.193:33581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dff7f1236b9643b9be0242601ff45f99: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
I20260809 05:39:10.803869 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.788266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.30.137.194:46597)
0809 05:39:10.788466 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.788474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.788496 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:39:10.788578 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.788583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.788669 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.788785 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:10.790407 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.791720 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.793354 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.793813 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.793968 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.797091 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:10.797105 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:10.797120 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:10.797154 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:10.800682 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.801451 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.801459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.801461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.801616 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.802050 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.802057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.802060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.802519 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:10.802927 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.803235 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.803492 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:39:10.804720 23351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:10.844312 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.843728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54690)
0809 05:39:10.843969 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.844104 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 05:39:10.845485 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.845071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54696)
0809 05:39:10.845234 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.845326 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 05:39:10.906281 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.905686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54710)
0809 05:39:10.905949 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.906072 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 05:39:10.907409 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.907031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54720)
0809 05:39:10.907191 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.907275 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 05:39:10.927259 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 23165
I20260809 05:39:10.955185 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 23361
I20260809 05:39:10.969086 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.956058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.30.137.195:38705)
0809 05:39:10.956260 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:10.956264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:10.956284 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:39:10.956503 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:10.956508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:10.956578 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:10.956685 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:10.958025 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.958903 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.960247 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:10.960645 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:10.960749 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:10.963418 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:10.963432 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:10.963448 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:10.963500 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:10.966454 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.967175 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.967183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.967186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.967332 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:10.967638 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:10.967641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:10.967643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:10.968006 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:10.968236 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:10.968514 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 05:39:10.968702 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 05:39:10.969825 23555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:10.988878 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.988308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54724)
0809 05:39:10.988525 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.988666 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 05:39:10.989984 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:10.989576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54734)
0809 05:39:10.989788 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:10.989868 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 05:39:11.063583 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 23361
I20260809 05:39:11.088876 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 22825
I20260809 05:39:11.090114 22916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:11.206900 23582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:11.206067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:33581 (local address 127.0.0.1:54742)
0809 05:39:11.206369 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:11.206522 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:39:11.206985 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 22825
I20260809 05:39:31.237560 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34445
--webserver_interface=127.30.137.254
--webserver_port=37405
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:34445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:31.604899 23635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:31.605302 23635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:31.605484 23635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:31.612914 23635 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 05:39:31.613081 23635 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 05:39:31.613176 23635 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 05:39:31.613279 23635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:39:31.613363 23635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:31.613490 23635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:31.613587 23635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:39:31.613674 23635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:39:31.624094 23635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:34445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=37405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:31.625556 23635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:31.627020 23635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:31.639549 23646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:31.639624 23640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:31.639611 23641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:31.640291 23635 server_base.cc:1034] running on GCE node
I20260809 05:39:31.641145 23635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:31.642557 23635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:31.643826 23635 hybrid_clock.cc:648] HybridClock initialized: now 1786253971643748 us; error 41 us; skew 500 ppm
I20260809 05:39:31.644186 23635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:31 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253971, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:31.654043 23635 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:39:31.655814 23635 webserver.cc:460] Webserver started at http://127.30.137.254:37405/ using document root <none> and password file <none>
I20260809 05:39:31.656612 23635 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:31.656764 23635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:31.662742 23635 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 05:39:31.665884 23659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:31.667411 23635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 05:39:31.667624 23635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "a6ce0cada05b43fe9b6b37ee413c6585"
format_stamp: "Formatted at 2026-08-09 05:38:45 on dist-test-slave-3kk6"
server_key: "f97fdffc6013344f99b8f612d9b73c6e"
server_key_iv: "0979adf0b3918a894c0b3e6f4da1aa90"
server_key_version: "encryptionkey@0"
I20260809 05:39:31.668709 23635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:31.682106 23635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:31.686887 23635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:31.687206 23635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:31.706732 23635 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:34445
I20260809 05:39:31.706753 23733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:34445 every 8 connection(s)
I20260809 05:39:31.708312 23635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:39:31.712209 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 23635
I20260809 05:39:31.713281 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:33581
--local_ip_for_outbound_sockets=127.30.137.193
--tserver_master_addrs=127.30.137.254:34445
--webserver_port=35461
--webserver_interface=127.30.137.193
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:39:31.717041 23734 sys_catalog.cc:263] Verifying existing consensus state
I20260809 05:39:31.722362 23734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap starting.
I20260809 05:39:31.744858 23734 log.cc:824] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:31.761518 23734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 05:39:31.762213 23734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Bootstrap complete.
I20260809 05:39:31.771620 23734 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:31.772670 23734 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ce0cada05b43fe9b6b37ee413c6585, State: Initialized, Role: FOLLOWER
I20260809 05:39:31.773366 23734 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:31.773618 23734 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:39:31.773770 23734 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:39:31.773980 23734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 2 FOLLOWER]: Advancing to term 3
I20260809 05:39:31.778064 23734 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:31.778635 23734 leader_election.cc:302] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6ce0cada05b43fe9b6b37ee413c6585; no voters:
I20260809 05:39:31.779436 23734 leader_election.cc:288] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 05:39:31.779590 23741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 05:39:31.780836 23741 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [term 3 LEADER]: Becoming Leader. State: Replica: a6ce0cada05b43fe9b6b37ee413c6585, State: Running, Role: LEADER
I20260809 05:39:31.781495 23741 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } }
I20260809 05:39:31.781749 23734 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:39:31.784121 23745 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6ce0cada05b43fe9b6b37ee413c6585. Latest consensus state: current_term: 3 leader_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } } }
I20260809 05:39:31.784571 23745 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:31.785200 23743 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ce0cada05b43fe9b6b37ee413c6585" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34445 } } }
I20260809 05:39:31.785526 23743 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:31.787822 23751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:39:31.798764 23751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:39:31.801164 23751 catalog_manager.cc:1259] Loaded cluster ID: fa4302127733423d95194e451793f5db
I20260809 05:39:31.801420 23751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:39:31.807195 23751 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:39:31.809695 23751 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Loaded TSK: 0
I20260809 05:39:31.810034 23751 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a6ce0cada05b43fe9b6b37ee413c6585: Loaded TSK: 1
F20260809 05:39:31.811733 23751 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f05cd5b29ad google::LogMessage::Fail() at ??:0
@ 0x7f05cd5b588a google::LogMessage::SendToLog() at ??:0
@ 0x7f05cd5b24de google::LogMessage::Flush() at ??:0
@ 0x7f05cd5b3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f05d13167d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f05d130573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f05e2fe11c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f05e2fe0783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f05e306c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f05e306c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f05e30b65e9 std::function<>::operator()() at ??:0
@ 0x7f05e2fe4cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f05e2fe2c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f05e306a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f05e306a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f05d01fe345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f05d020191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f05d0201799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f05d01dceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f05d34b66db start_thread at ??:0
@ 0x7f05cc44c61f clone at ??:0
W20260809 05:39:32.176998 23738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:32.177294 23738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:32.177393 23738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:32.184734 23738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:32.184852 23738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:32.184968 23738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:39:32.195055 23738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:33581
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=35461
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:32.196405 23738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:32.197759 23738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:32.212458 23809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:32.212497 23813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:32.212541 23811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:32.213840 23738 server_base.cc:1034] running on GCE node
I20260809 05:39:32.214449 23738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:32.215503 23738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:32.216796 23738 hybrid_clock.cc:648] HybridClock initialized: now 1786253972216745 us; error 36 us; skew 500 ppm
I20260809 05:39:32.217149 23738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:32 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253972, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:32.220922 23738 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:39:32.222496 23738 webserver.cc:460] Webserver started at http://127.30.137.193:35461/ using document root <none> and password file <none>
I20260809 05:39:32.223253 23738 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:32.223364 23738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:32.229014 23738 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 05:39:32.231899 23823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:32.233309 23738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 05:39:32.233517 23738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "a1457f307f3045c4ad8ea72d8ae584f8"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "6c51c439ac1d289e9c6237d1e8f02b6f"
server_key_iv: "598a54630d63ae9204712e080ca4a0f3"
server_key_version: "encryptionkey@0"
I20260809 05:39:32.234567 23738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:32.255857 23738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:32.260454 23738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:32.260771 23738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:32.261885 23738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:32.263803 23835 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260809 05:39:32.269290 23835 ts_tablet_manager.cc:1852] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 05:39:32.270314 23835 ts_tablet_manager.cc:1905] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:32.277644 23835 ts_tablet_manager.cc:1918] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:32.277958 23835 ts_tablet_manager.cc:1939] T dff7f1236b9643b9be0242601ff45f99 P a1457f307f3045c4ad8ea72d8ae584f8: Deleting consensus metadata
I20260809 05:39:32.278473 23738 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 05:39:32.278623 23738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260809 05:39:32.278851 23738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:32.278950 23738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:32.315691 23738 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:33581
I20260809 05:39:32.315778 23986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:33581 every 8 connection(s)
I20260809 05:39:32.317168 23738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:39:32.322042 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:32.320201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:60415)
0809 05:39:32.320972 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:32.321205 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":298,"threads_started":1}
W20260809 05:39:32.322685 23987 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:34445 (0 consecutive failures): Network error: Failed to ping master at 127.30.137.254:34445: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
I20260809 05:39:32.323197 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 23738
I20260809 05:39:32.323522 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:32.323182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:39983)
0809 05:39:32.323340 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:32.323410 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 05:39:32.323938 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:46597
--local_ip_for_outbound_sockets=127.30.137.194
--tserver_master_addrs=127.30.137.254:34445
--webserver_port=42741
--webserver_interface=127.30.137.194
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:39:32.324353 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:32.324015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:49663)
0809 05:39:32.324155 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:32.324233 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 05:39:32.324582 23987 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 05:39:32.678491 23990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:32.678905 23990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:32.679029 23990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:32.686504 23990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:32.686705 23990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:32.686863 23990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:39:32.697001 23990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:46597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=42741
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:32.698359 23990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:32.699821 23990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:32.713574 24001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:32.713546 23998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:32.713675 23997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:32.714447 23990 server_base.cc:1034] running on GCE node
I20260809 05:39:32.715167 23990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:32.716167 23990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:32.717403 23990 hybrid_clock.cc:648] HybridClock initialized: now 1786253972717348 us; error 55 us; skew 500 ppm
I20260809 05:39:32.717741 23990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:32 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253972, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:32.721201 23990 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:39:32.722841 23990 webserver.cc:460] Webserver started at http://127.30.137.194:42741/ using document root <none> and password file <none>
I20260809 05:39:32.723567 23990 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:32.723726 23990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:32.729359 23990 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 05:39:32.732378 24010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:32.733827 23990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:39:32.734025 23990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "7c3e2f1a5a324cb985896d007d0e6a32"
format_stamp: "Formatted at 2026-08-09 05:38:46 on dist-test-slave-3kk6"
server_key: "1b362ec57a26b3abca7462012c3a2210"
server_key_iv: "5adc0fb1815ad435fe0c44e17b91ecc4"
server_key_version: "encryptionkey@0"
I20260809 05:39:32.735150 23990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:32.751636 23990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:32.756019 23990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:32.756328 23990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:32.757424 23990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:32.758831 23990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:32.758965 23990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:32.759104 23990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:32.759202 23990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:32.795043 23990 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46597
I20260809 05:39:32.795120 24173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46597 every 8 connection(s)
I20260809 05:39:32.796648 23990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:39:32.801576 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:32.799724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:57557)
0809 05:39:32.800491 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:32.800748 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":301,"threads_started":1}
W20260809 05:39:32.802248 24174 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:34445 (0 consecutive failures): Network error: Failed to ping master at 127.30.137.254:34445: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
I20260809 05:39:32.803145 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:32.802816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:60023)
0809 05:39:32.802978 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:32.803044 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 05:39:32.803969 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:32.803628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:43523)
0809 05:39:32.803769 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:32.803857 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:39:32.804215 24174 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 05:39:32.807106 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 23990
I20260809 05:39:32.807750 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:38705
--local_ip_for_outbound_sockets=127.30.137.195
--tserver_master_addrs=127.30.137.254:34445
--webserver_port=40685
--webserver_interface=127.30.137.195
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:33.155318 24179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:33.155638 24179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:33.155743 24179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:33.163468 24179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:33.163599 24179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:33.163771 24179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:39:33.173763 24179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=40685
--tserver_master_addrs=127.30.137.254:34445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:33.175146 24179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:33.176429 24179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:33.189965 24189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:33.190086 24190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:33.190119 24193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:33.191023 24179 server_base.cc:1034] running on GCE node
I20260809 05:39:33.191618 24179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:33.192524 24179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:33.193794 24179 hybrid_clock.cc:648] HybridClock initialized: now 1786253973193748 us; error 55 us; skew 500 ppm
I20260809 05:39:33.194168 24179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:33 dist-test-slave-3kk6 krb5kdc[22012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253973, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:33.197817 24179 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:39:33.199502 24179 webserver.cc:460] Webserver started at http://127.30.137.195:40685/ using document root <none> and password file <none>
I20260809 05:39:33.200287 24179 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:33.200457 24179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:33.206107 24179 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 05:39:33.208992 24201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:33.210362 24179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:39:33.210552 24179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "18bc2c169dba426b809034133794a8e5"
format_stamp: "Formatted at 2026-08-09 05:38:47 on dist-test-slave-3kk6"
server_key: "2133b753e7458c73e2187894cf70b7fa"
server_key_iv: "8d632ed6c3e187a3624e0c86d96a4fa6"
server_key_version: "encryptionkey@0"
I20260809 05:39:33.211633 24179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:33.257541 24179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:33.262709 24179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:33.263019 24179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:33.264079 24179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:33.265600 24179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:33.265746 24179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:33.265873 24179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:33.265965 24179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:33.301075 24179 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:38705
I20260809 05:39:33.301167 24365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:38705 every 8 connection(s)
I20260809 05:39:33.302583 24179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:39:33.307543 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:33.305617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:39289)
0809 05:39:33.306359 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:33.306628 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":267,"threads_started":1}
W20260809 05:39:33.308276 24366 heartbeater.cc:644] Failed to heartbeat to 127.30.137.254:34445 (0 consecutive failures): Network error: Failed to ping master at 127.30.137.254:34445: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
I20260809 05:39:33.309123 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:33.308816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:43641)
0809 05:39:33.308950 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:33.309021 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 05:39:33.309904 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:33.309618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:42535)
0809 05:39:33.309732 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:33.309803 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 05:39:33.310156 24366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 05:39:33.310963 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 24179
I20260809 05:39:33.325971 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:33.325164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:33825)
0809 05:39:33.325664 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:33.325743 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":258,"threads_started":1}
I20260809 05:39:33.805630 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:33.804790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:36227)
0809 05:39:33.805286 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:33.805396 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":259,"threads_started":1}
I20260809 05:39:34.311748 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:34.310893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:58821)
0809 05:39:34.311397 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:34.311495 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":244,"threads_started":1}
I20260809 05:39:34.327747 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:34.326956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:46343)
0809 05:39:34.327488 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:34.327575 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":291,"threads_started":1}
I20260809 05:39:34.807536 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:34.806640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:53725)
0809 05:39:34.807255 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:34.807363 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":271,"threads_started":1}
I20260809 05:39:35.313467 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:35.312708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:58315)
0809 05:39:35.313217 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:35.313304 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
I20260809 05:39:35.329563 24384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:35.328792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:39279)
0809 05:39:35.329292 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:35.329395 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260809 05:39:35.809204 24385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:35.808399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:53313)
0809 05:39:35.808919 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:35.809011 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":286,"threads_started":1}
I20260809 05:39:36.315440 24386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:36.314434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:45173)
0809 05:39:36.315061 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:36.315175 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":311,"threads_started":1}
I20260809 05:39:36.331610 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:36.330819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:36105)
0809 05:39:36.331316 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:36.331419 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":242,"threads_started":1}
I20260809 05:39:36.811081 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:36.810247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:39211)
0809 05:39:36.810793 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:36.810879 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":253,"threads_started":1}
I20260809 05:39:37.317253 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:37.316389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:37451)
0809 05:39:37.316950 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:37.317054 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":240,"threads_started":1}
I20260809 05:39:37.333384 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:37.332626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:49597)
0809 05:39:37.333118 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:37.333224 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":263,"threads_started":1}
I20260809 05:39:37.812793 24395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:37.812050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:40063)
0809 05:39:37.812538 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:37.812621 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":262,"threads_started":1}
I20260809 05:39:38.319182 24397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:38.318321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:34371)
0809 05:39:38.318919 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:38.319010 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":347,"threads_started":1}
I20260809 05:39:38.335253 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:38.334374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:33667)
0809 05:39:38.334986 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:38.335073 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":369,"threads_started":1}
I20260809 05:39:38.814635 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:38.813786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:60197)
0809 05:39:38.814371 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:38.814467 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":298,"threads_started":1}
I20260809 05:39:39.320914 24401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:39.320127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:33965)
0809 05:39:39.320629 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:39.320742 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":272,"threads_started":1}
I20260809 05:39:39.337440 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:39.336332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:55911)
0809 05:39:39.337029 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:39.337134 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":346,"threads_started":1}
I20260809 05:39:39.816861 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:39.815810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:43123)
0809 05:39:39.816449 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:39.816616 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":298,"threads_started":1}
I20260809 05:39:40.322801 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:40.321936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:38673)
0809 05:39:40.322426 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:40.322524 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":222,"threads_started":1}
I20260809 05:39:40.339485 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:40.338530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:45425)
0809 05:39:40.339176 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:40.339270 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":411,"threads_started":1}
I20260809 05:39:40.818863 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:40.817996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:48029)
0809 05:39:40.818505 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:40.818598 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":263,"threads_started":1}
I20260809 05:39:41.324548 24408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:41.323772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:43403)
0809 05:39:41.324265 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:41.324366 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":259,"threads_started":1}
I20260809 05:39:41.341360 24410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:41.340460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:53125)
0809 05:39:41.341092 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:41.341190 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":322,"threads_started":1}
I20260809 05:39:41.820925 24411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:41.819898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:55087)
0809 05:39:41.820520 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:41.820663 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":307,"threads_started":1}
I20260809 05:39:42.326447 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:42.325599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:43245)
0809 05:39:42.326114 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:42.326234 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":263,"threads_started":1}
I20260809 05:39:42.343194 24416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:42.342406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:53015)
0809 05:39:42.342913 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:42.343004 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":298,"threads_started":1}
I20260809 05:39:42.822852 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:42.821995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:48321)
0809 05:39:42.822500 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:42.822590 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":284,"threads_started":1}
I20260809 05:39:43.328325 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:43.327579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:46685)
0809 05:39:43.328063 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:43.328160 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
I20260809 05:39:43.344974 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:43.344165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:53045)
0809 05:39:43.344678 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:43.344770 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":300,"threads_started":1}
I20260809 05:39:43.824649 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:43.823859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:40313)
0809 05:39:43.824370 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:43.824463 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":270,"threads_started":1}
I20260809 05:39:44.330184 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:44.329343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:34503)
0809 05:39:44.329817 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:44.329922 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":245,"threads_started":1}
I20260809 05:39:44.346745 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:44.345932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:34587)
0809 05:39:44.346420 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:44.346506 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":259,"threads_started":1}
I20260809 05:39:44.826536 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:44.825757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:37969)
0809 05:39:44.826264 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:44.826369 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
I20260809 05:39:45.331956 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:45.331244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:43827)
0809 05:39:45.331703 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:45.331792 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":227,"threads_started":1}
I20260809 05:39:45.348790 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:45.347874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:35033)
0809 05:39:45.348445 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:45.348606 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":329,"threads_started":1}
I20260809 05:39:45.828400 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:45.827570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:40313)
0809 05:39:45.828089 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:45.828206 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":260,"threads_started":1}
I20260809 05:39:46.333645 24430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:46.332883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:42295)
0809 05:39:46.333353 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:46.333452 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":240,"threads_started":1}
I20260809 05:39:46.350509 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:46.349746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:38307)
0809 05:39:46.350204 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:46.350307 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":235,"threads_started":1}
I20260809 05:39:46.830142 24433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:46.829348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:37473)
0809 05:39:46.829857 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:46.829953 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":267,"threads_started":1}
I20260809 05:39:47.335577 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:47.334763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:33501)
0809 05:39:47.335290 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:47.335392 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":265,"threads_started":1}
I20260809 05:39:47.352589 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:47.351775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.193:35027)
0809 05:39:47.352299 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:47.352428 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":243,"threads_started":1}
I20260809 05:39:47.832057 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:47.831315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.194:40183)
0809 05:39:47.831768 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:47.831869 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":231,"threads_started":1}
I20260809 05:39:48.314599 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 23738
I20260809 05:39:48.338177 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:48.336947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:34445 (local address 127.30.137.195:41253)
0809 05:39:48.337825 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:48.337951 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.254:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":512,"threads_started":1}
I20260809 05:39:48.339985 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:48.316115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.30.137.193:33581)
0809 05:39:48.317021 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:48.317041 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:48.317102 (+ 61us) server_negotiation.cc:407] Connection header received
0809 05:39:48.319196 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:48.319208 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:48.319677 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:48.320002 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:48.321647 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:48.323113 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:48.324421 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:48.325102 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:48.325246 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:48.332266 (+ 7020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:48.332276 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:48.332294 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:48.332351 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:48.335683 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:48.336694 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:48.336704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:48.336714 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:48.336960 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:48.337399 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:48.337406 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:48.337412 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:48.338151 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:48.338486 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:48.339122 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 05:39:48.339640 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":540,"threads_started":1}
I20260809 05:39:48.341598 23982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:48.427963 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 23738
I20260809 05:39:48.457929 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 23990
I20260809 05:39:48.476477 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:48.458947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.30.137.194:46597)
0809 05:39:48.459487 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:48.459501 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:48.459538 (+ 37us) server_negotiation.cc:407] Connection header received
0809 05:39:48.461144 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:48.461153 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:48.461581 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:48.461956 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:48.463778 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:48.465010 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:48.466347 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:48.467092 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:48.467294 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:48.470043 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:48.470051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:48.470066 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:48.470102 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:48.473038 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:48.473793 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:48.473800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:48.473806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:48.473974 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:48.474381 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:48.474386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:48.474389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:48.474964 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:48.475295 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:48.475774 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 05:39:48.476173 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":259,"threads_started":1}
I20260809 05:39:48.477887 24167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:48.563444 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 23990
I20260809 05:39:48.587141 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 24179
I20260809 05:39:48.606313 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:48.588070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.30.137.195:38705)
0809 05:39:48.588685 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:48.588699 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:48.588744 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:39:48.590489 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:48.590499 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:48.591032 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:48.591410 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:48.593080 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:48.594322 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:48.595773 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:48.596462 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:48.596669 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:48.599682 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:48.599690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:48.599705 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:48.599758 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:48.602850 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:48.603588 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:48.603595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:48.603602 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:48.603782 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:48.604202 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:48.604207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:48.604216 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:48.604769 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:48.605101 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:48.605572 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 05:39:48.605992 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":298,"threads_started":1}
I20260809 05:39:48.607839 24359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:48.692965 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 24179
2026-08-09T05:39:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65091 ms)
[----------] 30 tests from SecurityITest (312791 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:39:48 dist-test-slave-3kk6 krb5kdc[24456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:39:48 dist-test-slave-3kk6 krb5kdc[24456](info): set up 2 sockets
Aug 09 05:39:48 dist-test-slave-3kk6 krb5kdc[24456](info): commencing operation
krb5kdc: starting...
W20260809 05:39:50.028631 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:39:50 dist-test-slave-3kk6 krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:39:50.384043 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:40415
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:40415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:50.744069 24473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:50.744383 24473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:50.744477 24473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:50.751932 24473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:39:50.752058 24473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:50.752157 24473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:50.752271 24473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:39:50.752337 24473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:39:50.762426 24473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:40415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:40415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:50.763875 24473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:50.765285 24473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:50.777937 24479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:50.777966 24480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:50.778026 24483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:50.778707 24473 server_base.cc:1034] running on GCE node
I20260809 05:39:50.779680 24473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:50.781257 24473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:50.782511 24473 hybrid_clock.cc:648] HybridClock initialized: now 1786253990782466 us; error 75 us; skew 500 ppm
I20260809 05:39:50.782929 24473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:50 dist-test-slave-3kk6 krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253990, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:50.786479 24473 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:39:50.788167 24473 webserver.cc:460] Webserver started at http://127.30.137.254:37871/ using document root <none> and password file <none>
I20260809 05:39:50.788959 24473 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:50.789122 24473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:50.789458 24473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:50.792894 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "a16d921a41164105b4b0a8960c1e9173"
format_stamp: "Formatted at 2026-08-09 05:39:50 on dist-test-slave-3kk6"
server_key: "ea8e217684b754a344a2647e84eca805"
server_key_iv: "5fb2686f4a89a808a54249578be11056"
server_key_version: "encryptionkey@0"
I20260809 05:39:50.793728 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "a16d921a41164105b4b0a8960c1e9173"
format_stamp: "Formatted at 2026-08-09 05:39:50 on dist-test-slave-3kk6"
server_key: "ea8e217684b754a344a2647e84eca805"
server_key_iv: "5fb2686f4a89a808a54249578be11056"
server_key_version: "encryptionkey@0"
I20260809 05:39:50.799297 24473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:39:50.802917 24491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:50.804221 24473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:39:50.804425 24473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "a16d921a41164105b4b0a8960c1e9173"
format_stamp: "Formatted at 2026-08-09 05:39:50 on dist-test-slave-3kk6"
server_key: "ea8e217684b754a344a2647e84eca805"
server_key_iv: "5fb2686f4a89a808a54249578be11056"
server_key_version: "encryptionkey@0"
I20260809 05:39:50.804636 24473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:50.837329 24473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:50.841810 24473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:50.842155 24473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:50.861871 24473 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:40415
I20260809 05:39:50.861893 24558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:40415 every 8 connection(s)
I20260809 05:39:50.863390 24473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:39:50.867362 24561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:50.870483 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 24473
I20260809 05:39:50.870798 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:39:50.871698 31271 external_mini_cluster.cc:1428] Setting key c0a40b5cae9d7e896e884e54aec6822f
I20260809 05:39:50.881083 24561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: Bootstrap starting.
I20260809 05:39:50.885275 24561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:50.886751 24561 log.cc:824] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:39:50 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:50.890394 24561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: No bootstrap required, opened a new log
I20260809 05:39:50.898065 24564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:50.874019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.30.137.254:40415)
0809 05:39:50.874878 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:50.874890 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:50.874932 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:39:50.876740 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:50.876764 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:50.877197 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:50.877541 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:50.879538 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:50.880911 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:50.882970 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:50.883681 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:50.883821 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:50.889666 (+ 5845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:50.889675 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:50.889690 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:50.889728 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:50.893366 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:50.894388 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:50.894394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:50.894401 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:50.894596 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:50.895107 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:50.895111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:50.895113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:50.895733 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:50.895976 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:50.896500 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 05:39:50.896927 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":403,"threads_started":1}
I20260809 05:39:50.900243 24561 raft_consensus.cc:348] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16d921a41164105b4b0a8960c1e9173" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 40415 } }
I20260809 05:39:50.900766 24561 raft_consensus.cc:374] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:50.900892 24561 raft_consensus.cc:729] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16d921a41164105b4b0a8960c1e9173, State: Initialized, Role: FOLLOWER
I20260809 05:39:50.901533 24561 consensus_queue.cc:260] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16d921a41164105b4b0a8960c1e9173" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 40415 } }
I20260809 05:39:50.901767 24561 raft_consensus.cc:388] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:39:50.901871 24561 raft_consensus.cc:482] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:39:50.902004 24561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:50.904359 24561 raft_consensus.cc:504] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16d921a41164105b4b0a8960c1e9173" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 40415 } }
I20260809 05:39:50.904934 24561 leader_election.cc:302] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a16d921a41164105b4b0a8960c1e9173; no voters:
I20260809 05:39:50.905501 24561 leader_election.cc:288] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:39:50.905661 24566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:39:50.906826 24566 raft_consensus.cc:686] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [term 1 LEADER]: Becoming Leader. State: Replica: a16d921a41164105b4b0a8960c1e9173, State: Running, Role: LEADER
I20260809 05:39:50.907366 24566 consensus_queue.cc:237] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16d921a41164105b4b0a8960c1e9173" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 40415 } }
I20260809 05:39:50.907788 24561 sys_catalog.cc:564] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:39:50.910728 24570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a16d921a41164105b4b0a8960c1e9173. Latest consensus state: current_term: 1 leader_uuid: "a16d921a41164105b4b0a8960c1e9173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16d921a41164105b4b0a8960c1e9173" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 40415 } } }
I20260809 05:39:50.911137 24569 sys_catalog.cc:455] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a16d921a41164105b4b0a8960c1e9173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16d921a41164105b4b0a8960c1e9173" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 40415 } } }
I20260809 05:39:50.911623 24570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:50.911798 24569 sys_catalog.cc:458] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173 [sys.catalog]: This master's current role is: LEADER
W20260809 05:39:50.922432 24587 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:39:50.922606 24587 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:39:50.922818 24575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:39:50.924711 24575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:39:50.932829 24575 catalog_manager.cc:1347] Generated new cluster ID: 562a890909e9413eb3919fe4f686a71a
I20260809 05:39:50.933015 24575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:39:50.950873 24575 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:39:50.952961 24575 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:39:50.979909 24575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: Generated new TSK 0
I20260809 05:39:50.980891 24575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:39:51.061911 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:40415
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:51.428179 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:51.428478 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:51.428607 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:51.436118 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:51.436275 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:51.436468 24595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:39:51.446733 24595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:40415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:51.448153 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:51.449502 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:51.463627 24603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:51.463675 24602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:51.464468 24605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:51.464424 24595 server_base.cc:1034] running on GCE node
I20260809 05:39:51.465274 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:51.466246 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:51.467454 24595 hybrid_clock.cc:648] HybridClock initialized: now 1786253991467415 us; error 33 us; skew 500 ppm
I20260809 05:39:51.467760 24595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:51 dist-test-slave-3kk6 krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253991, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:51.471379 24595 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:39:51.472997 24595 webserver.cc:460] Webserver started at http://127.30.137.193:37565/ using document root <none> and password file <none>
I20260809 05:39:51.473745 24595 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:51.473906 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:51.474256 24595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:51.477468 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "76ff9345b7664a889c117a947a0a9ee2"
format_stamp: "Formatted at 2026-08-09 05:39:51 on dist-test-slave-3kk6"
server_key: "db35f313f8b6939249d761c2dd5ae0b0"
server_key_iv: "e6baf3cb86396166d448d96bc4150517"
server_key_version: "encryptionkey@0"
I20260809 05:39:51.478295 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "76ff9345b7664a889c117a947a0a9ee2"
format_stamp: "Formatted at 2026-08-09 05:39:51 on dist-test-slave-3kk6"
server_key: "db35f313f8b6939249d761c2dd5ae0b0"
server_key_iv: "e6baf3cb86396166d448d96bc4150517"
server_key_version: "encryptionkey@0"
I20260809 05:39:51.483824 24595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 05:39:51.487509 24615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:51.488901 24595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 05:39:51.489109 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "76ff9345b7664a889c117a947a0a9ee2"
format_stamp: "Formatted at 2026-08-09 05:39:51 on dist-test-slave-3kk6"
server_key: "db35f313f8b6939249d761c2dd5ae0b0"
server_key_iv: "e6baf3cb86396166d448d96bc4150517"
server_key_version: "encryptionkey@0"
I20260809 05:39:51.489336 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:51.512741 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:51.517266 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:51.517584 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:51.518698 24595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:51.520143 24595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:51.520227 24595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:51.520382 24595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:51.520452 24595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:51.556087 24595 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:46859
I20260809 05:39:51.556173 24761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:46859 every 8 connection(s)
I20260809 05:39:51.557489 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:39:51.567490 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 24595
I20260809 05:39:51.567726 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:39:51.568452 31271 external_mini_cluster.cc:1428] Setting key f11fd939d29cb9b863fd4be8f770ca9a
Aug 09 05:39:51 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253991, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:51.581169 24764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:51.560183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:51515 (local address 127.30.137.254:40415)
0809 05:39:51.560706 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:51.560710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:51.561581 (+ 871us) server_negotiation.cc:407] Connection header received
0809 05:39:51.563601 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:51.563609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:51.563702 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:51.564029 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:51.565815 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:51.566587 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:51.568473 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:51.568978 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:51.569132 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:51.573505 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:51.573514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:51.573522 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:51.573577 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:51.577374 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:51.578226 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:51.578234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:51.578240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:51.578410 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:51.578949 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:51.578956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:51.578961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:51.579366 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:51.579533 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:51.580373 (+ 840us) server_negotiation.cc:299] Negotiation successful
0809 05:39:51.580660 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
I20260809 05:39:51.581871 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:51.560626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:40415 (local address 127.30.137.193:51515)
0809 05:39:51.561381 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:51.561429 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:51.563296 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:51.564312 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:51.564321 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:51.564950 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:51.565570 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:51.565589 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:51.566828 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:51.566841 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:51.568294 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:51.568309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:51.569230 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:51.569235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:51.569557 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:51.570370 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:39:51.570394 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:51.573218 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:51.577545 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:51.577553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:51.577568 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:51.578089 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:51.578532 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:51.578536 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:51.578538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:51.578829 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:51.579517 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:51.579523 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:51.579937 (+ 414us) client_negotiation.cc:769] Sending connection context
0809 05:39:51.580189 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 05:39:51.580635 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":294,"threads_started":1}
I20260809 05:39:51.584101 24762 heartbeater.cc:347] Connected to a master server at 127.30.137.254:40415
I20260809 05:39:51.584532 24762 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:51.585372 24762 heartbeater.cc:511] Master 127.30.137.254:40415 requested a full tablet report, sending...
I20260809 05:39:51.588162 24510 ts_manager.cc:194] Registered new tserver with Master: 76ff9345b7664a889c117a947a0a9ee2 (127.30.137.193:46859)
I20260809 05:39:51.590137 24510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:51515
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:39:51.648352 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:40415
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:52.006248 24768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:52.006640 24768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:52.006851 24768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:52.014781 24768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:52.014950 24768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:52.015167 24768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:39:52.026139 24768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:40415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:52.027683 24768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:52.029098 24768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:52.043442 24775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:52.043470 24777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:52.043674 24780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:52.044340 24768 server_base.cc:1034] running on GCE node
I20260809 05:39:52.045243 24768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:52.046484 24768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:52.047749 24768 hybrid_clock.cc:648] HybridClock initialized: now 1786253992047675 us; error 63 us; skew 500 ppm
I20260809 05:39:52.048115 24768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:52 dist-test-slave-3kk6 krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:52.051548 24768 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:39:52.053215 24768 webserver.cc:460] Webserver started at http://127.30.137.194:46657/ using document root <none> and password file <none>
I20260809 05:39:52.053957 24768 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:52.054116 24768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:52.054450 24768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:52.057650 24768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "076b517be7f44f42ad12ff04c4e112d6"
format_stamp: "Formatted at 2026-08-09 05:39:52 on dist-test-slave-3kk6"
server_key: "853742cfa5a6e322686c5e6244843985"
server_key_iv: "81d0306f50e010610d2d4f598261f3d7"
server_key_version: "encryptionkey@0"
I20260809 05:39:52.058486 24768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "076b517be7f44f42ad12ff04c4e112d6"
format_stamp: "Formatted at 2026-08-09 05:39:52 on dist-test-slave-3kk6"
server_key: "853742cfa5a6e322686c5e6244843985"
server_key_iv: "81d0306f50e010610d2d4f598261f3d7"
server_key_version: "encryptionkey@0"
I20260809 05:39:52.064531 24768 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 05:39:52.068364 24790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:52.070073 24768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:39:52.070227 24768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "076b517be7f44f42ad12ff04c4e112d6"
format_stamp: "Formatted at 2026-08-09 05:39:52 on dist-test-slave-3kk6"
server_key: "853742cfa5a6e322686c5e6244843985"
server_key_iv: "81d0306f50e010610d2d4f598261f3d7"
server_key_version: "encryptionkey@0"
I20260809 05:39:52.070434 24768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:52.106026 24768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:52.112859 24768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:52.113193 24768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:52.114645 24768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:52.116727 24768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:52.116839 24768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:52.116994 24768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:52.117072 24768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:52.161710 24768 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46547
I20260809 05:39:52.161765 24944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46547 every 8 connection(s)
I20260809 05:39:52.163832 24768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:39:52.164149 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 24768
I20260809 05:39:52.164361 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:39:52.164902 31271 external_mini_cluster.cc:1428] Setting key af1d68e58f8cc908424674486eae13af
Aug 09 05:39:52 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:52.190832 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.167052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:55465 (local address 127.30.137.254:40415)
0809 05:39:52.167580 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:52.167589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:52.168957 (+ 1368us) server_negotiation.cc:407] Connection header received
0809 05:39:52.172176 (+ 3219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:52.172185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:52.172292 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:52.172431 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:52.175065 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.176534 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.179510 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.180009 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.180182 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.184403 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:52.184409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:52.184412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:52.184449 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:52.187398 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:52.188226 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:52.188231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:52.188239 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:52.188383 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:52.188874 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:52.188881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:52.188889 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:52.189182 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:52.189365 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:52.190138 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 05:39:52.190399 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":232,"threads_started":1}
I20260809 05:39:52.191929 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.167825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:40415 (local address 127.30.137.194:55465)
0809 05:39:52.168754 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:52.168813 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:52.171705 (+ 2892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:52.172811 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:52.172828 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:52.173767 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:52.174732 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.174767 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.176786 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.176795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.179273 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.179296 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.180303 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.180313 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.180569 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.181388 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:39:52.181418 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:52.184099 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:52.187570 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:52.187580 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:52.187598 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:52.188075 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:52.188513 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:52.188518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:52.188522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:52.188739 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:52.189329 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:52.189338 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:52.189722 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 05:39:52.190033 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 05:39:52.190619 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":377,"threads_started":1}
I20260809 05:39:52.193547 24947 heartbeater.cc:347] Connected to a master server at 127.30.137.254:40415
I20260809 05:39:52.193931 24947 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:52.194959 24947 heartbeater.cc:511] Master 127.30.137.254:40415 requested a full tablet report, sending...
I20260809 05:39:52.196751 24510 ts_manager.cc:194] Registered new tserver with Master: 076b517be7f44f42ad12ff04c4e112d6 (127.30.137.194:46547)
I20260809 05:39:52.198117 24510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:55465
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:39:52.245631 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:40415
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:39181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:39:52.597500 24762 heartbeater.cc:503] Master 127.30.137.254:40415 was elected leader, sending a full tablet report...
W20260809 05:39:52.613965 24953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:52.614270 24953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:52.614454 24953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:52.622200 24953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:52.622323 24953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:52.622437 24953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:39:52.632519 24953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:39181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:40415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:52.633890 24953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:52.635272 24953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:52.649327 24962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:52.649264 24963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:52.649533 24965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:52.650153 24953 server_base.cc:1034] running on GCE node
I20260809 05:39:52.650910 24953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:52.651975 24953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:52.653183 24953 hybrid_clock.cc:648] HybridClock initialized: now 1786253992653136 us; error 45 us; skew 500 ppm
I20260809 05:39:52.653524 24953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:52 dist-test-slave-3kk6 krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:52.657029 24953 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:39:52.658638 24953 webserver.cc:460] Webserver started at http://127.30.137.195:40723/ using document root <none> and password file <none>
I20260809 05:39:52.659444 24953 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:52.659617 24953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:52.660044 24953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:52.663306 24953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "ce0e5834fff4464b8d0650bf156ace12"
format_stamp: "Formatted at 2026-08-09 05:39:52 on dist-test-slave-3kk6"
server_key: "63ec0da1c32d784ec1479d793a98a63b"
server_key_iv: "d8c3cdabf40968361e8e31c0a9a86776"
server_key_version: "encryptionkey@0"
I20260809 05:39:52.664106 24953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "ce0e5834fff4464b8d0650bf156ace12"
format_stamp: "Formatted at 2026-08-09 05:39:52 on dist-test-slave-3kk6"
server_key: "63ec0da1c32d784ec1479d793a98a63b"
server_key_iv: "d8c3cdabf40968361e8e31c0a9a86776"
server_key_version: "encryptionkey@0"
I20260809 05:39:52.669641 24953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:39:52.673233 24974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:52.674773 24953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:39:52.674957 24953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "ce0e5834fff4464b8d0650bf156ace12"
format_stamp: "Formatted at 2026-08-09 05:39:52 on dist-test-slave-3kk6"
server_key: "63ec0da1c32d784ec1479d793a98a63b"
server_key_iv: "d8c3cdabf40968361e8e31c0a9a86776"
server_key_version: "encryptionkey@0"
I20260809 05:39:52.675174 24953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:52.706905 24953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:52.711637 24953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:52.711982 24953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:52.713052 24953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:52.714493 24953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:52.714572 24953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:52.714748 24953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:52.714813 24953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:52.750900 24953 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:40721
I20260809 05:39:52.750936 25124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:40721 every 8 connection(s)
I20260809 05:39:52.752337 24953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:39:52.761668 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 24953
I20260809 05:39:52.761871 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:39:52.762382 31271 external_mini_cluster.cc:1428] Setting key 49c6278be9075264eb6db75310b28c11
Aug 09 05:39:52 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:52.773701 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.755321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:57125 (local address 127.30.137.254:40415)
0809 05:39:52.755895 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:52.755899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:52.756717 (+ 818us) server_negotiation.cc:407] Connection header received
0809 05:39:52.758615 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:52.758621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:52.758747 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:52.758873 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:52.760470 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.761282 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.763280 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.763897 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.764061 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.767979 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:52.767984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:52.767988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:52.768027 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:52.770617 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:52.771467 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:52.771471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:52.771473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:52.771601 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:52.772048 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:52.772051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:52.772053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:52.772333 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:52.772451 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:52.773178 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 05:39:52.773364 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":343,"threads_started":1}
I20260809 05:39:52.774910 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.755783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:40415 (local address 127.30.137.195:57125)
0809 05:39:52.756526 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:52.756567 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:52.758323 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:52.759135 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:52.759143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:52.759683 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:52.760267 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.760285 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.761451 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.761462 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.763079 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.763096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.764136 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.764141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.764355 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.765192 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:39:52.765218 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:52.767755 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:52.770859 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:52.770879 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:52.770894 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:52.771353 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:52.771719 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:39:52.771723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:39:52.771725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:39:52.771946 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:39:52.772445 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:52.772452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:52.772810 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 05:39:52.773083 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 05:39:52.773514 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":304,"threads_started":1}
I20260809 05:39:52.776684 25125 heartbeater.cc:347] Connected to a master server at 127.30.137.254:40415
I20260809 05:39:52.777086 25125 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:52.778043 25125 heartbeater.cc:511] Master 127.30.137.254:40415 requested a full tablet report, sending...
I20260809 05:39:52.779871 24510 ts_manager.cc:194] Registered new tserver with Master: ce0e5834fff4464b8d0650bf156ace12 (127.30.137.195:40721)
I20260809 05:39:52.780835 24510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:57125
I20260809 05:39:52.791481 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:39:52.810146 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.796377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.30.137.254:40415)
0809 05:39:52.796560 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:52.796564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:52.797116 (+ 552us) server_negotiation.cc:407] Connection header received
0809 05:39:52.797323 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:52.797327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:52.797410 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:52.797504 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:52.799151 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.799918 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.801243 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.801641 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.801737 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.804338 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:52.804343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:52.804347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:52.804381 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:52.807275 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:52.808154 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:52.808158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:52.808160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:52.808296 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:52.808655 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:52.808658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:52.808660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:52.809104 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:52.809289 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:52.809638 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 05:39:52.809908 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 05:39:52.814237 24510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:39:52.821614 24510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:39:52.856396 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.835020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50182 (local address 127.30.137.195:40721)
0809 05:39:52.848183 (+ 13163us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:52.848193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:52.848235 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:39:52.848360 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:52.848367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:52.848573 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:52.848778 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:52.849575 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.851259 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.853468 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.854633 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.854880 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.855078 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:52.855783 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 05:39:52.855956 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12964,"thread_start_us":12772,"threads_started":1}
I20260809 05:39:52.856601 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.833833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:46859 (local address 127.0.0.1:36352)
0809 05:39:52.846674 (+ 12841us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:52.846739 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:52.846935 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:52.847938 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:52.847946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:52.847978 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:52.848492 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.848507 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.850560 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.850565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.853260 (+ 2695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.853276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.855639 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.855647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.855857 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.855894 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:39:52.855979 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 05:39:52.856125 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12468,"mutex_wait_us":58,"thread_start_us":13300,"threads_started":1}
I20260809 05:39:52.856805 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.834191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.30.137.193:46859)
0809 05:39:52.846991 (+ 12800us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:52.847002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:52.847035 (+ 33us) server_negotiation.cc:407] Connection header received
0809 05:39:52.847387 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:52.847398 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:52.847674 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:52.847922 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:52.848720 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.850432 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.853479 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.855048 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.855260 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.855434 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:52.856276 (+ 842us) server_negotiation.cc:299] Negotiation successful
0809 05:39:52.856458 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12583,"thread_start_us":12864,"threads_started":1}
I20260809 05:39:52.857446 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.834418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46547 (local address 127.0.0.1:34438)
0809 05:39:52.848630 (+ 14212us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:52.848649 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:52.848773 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:52.849679 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:52.849688 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:52.849713 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:52.850120 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.850132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.851717 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.851725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.854019 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.854034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.856269 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.856277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.856458 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.856969 (+ 511us) client_negotiation.cc:769] Sending connection context
0809 05:39:52.857062 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 05:39:52.857172 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14072,"mutex_wait_us":1,"thread_start_us":11681,"threads_started":1}
I20260809 05:39:52.858122 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.835818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.30.137.194:46547)
0809 05:39:52.847812 (+ 11994us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:52.847822 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:52.848921 (+ 1099us) server_negotiation.cc:407] Connection header received
0809 05:39:52.849040 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:52.849047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:52.849248 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:52.849447 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:52.850309 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.851539 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.854238 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.855748 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.855929 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.856091 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:52.857723 (+ 1632us) server_negotiation.cc:299] Negotiation successful
0809 05:39:52.857883 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11804,"thread_start_us":11692,"threads_started":1}
I20260809 05:39:52.858119 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:52.834186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:40721 (local address 127.0.0.1:50182)
0809 05:39:52.847370 (+ 13184us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:52.847390 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:52.847533 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:52.848890 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:52.848896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:52.848926 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:52.849327 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.849341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.851428 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.851433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.853267 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:52.853280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:52.855230 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:52.855236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:52.855457 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:52.855518 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 05:39:52.855710 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 05:39:52.855951 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13013,"mutex_wait_us":76,"thread_start_us":14435,"threads_started":1}
I20260809 05:39:52.861420 25042 tablet_service.cc:1467] Processing CreateTablet for tablet a5f118bf4c2048ccb05718f610b9fae0 (DEFAULT_TABLE table=test-table [id=b0b860d347bb424ca418886e3cf65ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:52.861831 24858 tablet_service.cc:1467] Processing CreateTablet for tablet a5f118bf4c2048ccb05718f610b9fae0 (DEFAULT_TABLE table=test-table [id=b0b860d347bb424ca418886e3cf65ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:52.863170 24858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f118bf4c2048ccb05718f610b9fae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:52.863188 25042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f118bf4c2048ccb05718f610b9fae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:52.861088 24676 tablet_service.cc:1467] Processing CreateTablet for tablet a5f118bf4c2048ccb05718f610b9fae0 (DEFAULT_TABLE table=test-table [id=b0b860d347bb424ca418886e3cf65ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:52.864449 24676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5f118bf4c2048ccb05718f610b9fae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:52.884321 25150 tablet_bootstrap.cc:492] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Bootstrap starting.
I20260809 05:39:52.885447 25154 tablet_bootstrap.cc:492] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Bootstrap starting.
I20260809 05:39:52.888618 25155 tablet_bootstrap.cc:492] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Bootstrap starting.
I20260809 05:39:52.889204 25154 tablet_bootstrap.cc:654] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:52.889456 25150 tablet_bootstrap.cc:654] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:52.890611 25154 log.cc:824] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:52.890805 25150 log.cc:824] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:52.892356 25155 tablet_bootstrap.cc:654] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:52.893649 25155 log.cc:824] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:52.894306 25150 tablet_bootstrap.cc:492] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: No bootstrap required, opened a new log
I20260809 05:39:52.894307 25154 tablet_bootstrap.cc:492] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: No bootstrap required, opened a new log
I20260809 05:39:52.894604 25150 ts_tablet_manager.cc:1397] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 05:39:52.894604 25154 ts_tablet_manager.cc:1397] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 05:39:52.896569 25155 tablet_bootstrap.cc:492] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: No bootstrap required, opened a new log
I20260809 05:39:52.896855 25155 ts_tablet_manager.cc:1397] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 05:39:52.904623 25155 raft_consensus.cc:348] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:52.905053 25155 raft_consensus.cc:374] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:52.905157 25155 raft_consensus.cc:729] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ff9345b7664a889c117a947a0a9ee2, State: Initialized, Role: FOLLOWER
I20260809 05:39:52.905805 25155 consensus_queue.cc:260] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:52.905926 25154 raft_consensus.cc:348] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:52.905926 25150 raft_consensus.cc:348] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:52.906399 25154 raft_consensus.cc:374] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:52.906399 25150 raft_consensus.cc:374] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:52.906519 25150 raft_consensus.cc:729] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076b517be7f44f42ad12ff04c4e112d6, State: Initialized, Role: FOLLOWER
I20260809 05:39:52.906513 25154 raft_consensus.cc:729] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0e5834fff4464b8d0650bf156ace12, State: Initialized, Role: FOLLOWER
I20260809 05:39:52.907198 25150 consensus_queue.cc:260] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:52.907240 25154 consensus_queue.cc:260] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:52.908159 25155 ts_tablet_manager.cc:1428] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 05:39:52.909626 24947 heartbeater.cc:503] Master 127.30.137.254:40415 was elected leader, sending a full tablet report...
I20260809 05:39:52.909644 25125 heartbeater.cc:503] Master 127.30.137.254:40415 was elected leader, sending a full tablet report...
I20260809 05:39:52.910272 25150 ts_tablet_manager.cc:1428] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260809 05:39:52.911119 25154 ts_tablet_manager.cc:1428] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260809 05:39:52.916344 24948 tablet.cc:2369] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:39:53.004271 25126 tablet.cc:2369] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:39:53.060365 24763 tablet.cc:2369] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:39:53.143007 25161 raft_consensus.cc:482] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:39:53.143414 25161 raft_consensus.cc:504] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:53.145198 25161 leader_election.cc:288] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76ff9345b7664a889c117a947a0a9ee2 (127.30.137.193:46859), 076b517be7f44f42ad12ff04c4e112d6 (127.30.137.194:46547)
I20260809 05:39:53.151048 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.145759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:44527 (local address 127.30.137.194:46547)
0809 05:39:53.145997 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:53.146005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:53.146031 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:39:53.146119 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:53.146126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:53.146211 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:53.146327 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:53.146825 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.147582 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.149241 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.150052 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.150233 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.150310 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:53.150611 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 05:39:53.150778 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 05:39:53.151049 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.145626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46547 (local address 127.30.137.195:44527)
0809 05:39:53.145827 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:53.145848 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:53.145985 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:53.146392 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:53.146398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:53.146432 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:53.146622 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:53.146631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.147732 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.147739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:53.149046 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:53.149060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.150234 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.150240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:53.150435 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.150462 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 05:39:53.150578 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 05:39:53.150745 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 05:39:53.152063 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.146045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:38539 (local address 127.30.137.193:46859)
0809 05:39:53.146256 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:53.146264 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:53.146753 (+ 489us) server_negotiation.cc:407] Connection header received
0809 05:39:53.146976 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:53.146980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:53.147041 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:53.147149 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:53.147708 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.148463 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.150185 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.151051 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.151244 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.151321 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:53.151658 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 05:39:53.151797 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:39:53.152155 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.145626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:46859 (local address 127.30.137.195:38539)
0809 05:39:53.146602 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:53.146616 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:53.146807 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:53.147167 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:53.147172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:53.147198 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:53.147543 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:53.147552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.148639 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.148643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:53.150003 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:53.150015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.151280 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.151286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:53.151468 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.151500 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:39:53.151598 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 05:39:53.151749 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":252,"thread_start_us":373,"threads_started":1}
I20260809 05:39:53.152310 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f118bf4c2048ccb05718f610b9fae0" candidate_uuid: "ce0e5834fff4464b8d0650bf156ace12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076b517be7f44f42ad12ff04c4e112d6" is_pre_election: true
I20260809 05:39:53.152992 24888 raft_consensus.cc:2454] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0e5834fff4464b8d0650bf156ace12 in term 0.
I20260809 05:39:53.153322 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f118bf4c2048ccb05718f610b9fae0" candidate_uuid: "ce0e5834fff4464b8d0650bf156ace12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ff9345b7664a889c117a947a0a9ee2" is_pre_election: true
I20260809 05:39:53.153797 24976 leader_election.cc:302] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076b517be7f44f42ad12ff04c4e112d6, ce0e5834fff4464b8d0650bf156ace12; no voters:
I20260809 05:39:53.153882 24702 raft_consensus.cc:2454] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0e5834fff4464b8d0650bf156ace12 in term 0.
I20260809 05:39:53.154249 25161 raft_consensus.cc:2781] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:39:53.154453 25161 raft_consensus.cc:482] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:39:53.154597 25161 raft_consensus.cc:3037] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:53.156962 25161 raft_consensus.cc:504] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:53.157568 25161 leader_election.cc:288] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [CANDIDATE]: Term 1 election: Requested vote from peers 76ff9345b7664a889c117a947a0a9ee2 (127.30.137.193:46859), 076b517be7f44f42ad12ff04c4e112d6 (127.30.137.194:46547)
I20260809 05:39:53.158130 24888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f118bf4c2048ccb05718f610b9fae0" candidate_uuid: "ce0e5834fff4464b8d0650bf156ace12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "076b517be7f44f42ad12ff04c4e112d6"
I20260809 05:39:53.158130 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5f118bf4c2048ccb05718f610b9fae0" candidate_uuid: "ce0e5834fff4464b8d0650bf156ace12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76ff9345b7664a889c117a947a0a9ee2"
I20260809 05:39:53.158352 24888 raft_consensus.cc:3037] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:53.158370 24702 raft_consensus.cc:3037] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:53.161793 24702 raft_consensus.cc:2454] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0e5834fff4464b8d0650bf156ace12 in term 1.
I20260809 05:39:53.161803 24888 raft_consensus.cc:2454] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0e5834fff4464b8d0650bf156ace12 in term 1.
I20260809 05:39:53.162468 24976 leader_election.cc:302] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 076b517be7f44f42ad12ff04c4e112d6, ce0e5834fff4464b8d0650bf156ace12; no voters:
I20260809 05:39:53.162828 25161 raft_consensus.cc:2781] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:39:53.163371 25161 raft_consensus.cc:686] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 1 LEADER]: Becoming Leader. State: Replica: ce0e5834fff4464b8d0650bf156ace12, State: Running, Role: LEADER
I20260809 05:39:53.163918 25161 consensus_queue.cc:237] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } }
I20260809 05:39:53.169050 24509 catalog_manager.cc:5495] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0e5834fff4464b8d0650bf156ace12 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "ce0e5834fff4464b8d0650bf156ace12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0e5834fff4464b8d0650bf156ace12" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 40721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 } health_report { overall_health: UNKNOWN } } }
I20260809 05:39:53.219698 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.214289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.30.137.195:40721)
0809 05:39:53.214484 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:53.214489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:53.214510 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:39:53.214603 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:53.214607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:53.214754 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:53.214893 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:39:53.215246 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.216039 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.217569 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.217979 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.218074 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.218542 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:39:53.219145 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:39:53.219258 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:53.219332 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 05:39:53.219397 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:39:53.227641 24888 raft_consensus.cc:1261] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 1 FOLLOWER]: Refusing update from remote peer ce0e5834fff4464b8d0650bf156ace12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:39:53.227689 24702 raft_consensus.cc:1261] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 1 FOLLOWER]: Refusing update from remote peer ce0e5834fff4464b8d0650bf156ace12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:39:53.228657 25165 consensus_queue.cc:1035] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76ff9345b7664a889c117a947a0a9ee2" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:39:53.229023 25161 consensus_queue.cc:1035] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "076b517be7f44f42ad12ff04c4e112d6" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:39:53.256882 24509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:39:53.257167 24509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:39:53.260797 24509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a16d921a41164105b4b0a8960c1e9173: Sending DeleteTablet for 3 replicas of tablet a5f118bf4c2048ccb05718f610b9fae0
I20260809 05:39:53.261973 24676 tablet_service.cc:1514] Processing DeleteTablet for tablet a5f118bf4c2048ccb05718f610b9fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:53 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36352
I20260809 05:39:53.262300 25042 tablet_service.cc:1514] Processing DeleteTablet for tablet a5f118bf4c2048ccb05718f610b9fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:53 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:50182
I20260809 05:39:53.262804 24858 tablet_service.cc:1514] Processing DeleteTablet for tablet a5f118bf4c2048ccb05718f610b9fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:53 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:34438
I20260809 05:39:53.263408 25184 tablet_replica.cc:330] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: stopping tablet replica
I20260809 05:39:53.264010 25184 raft_consensus.cc:2229] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:39:53.267930 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 24595
I20260809 05:39:53.273461 25190 tablet_replica.cc:330] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: stopping tablet replica
I20260809 05:39:53.274093 25190 raft_consensus.cc:2229] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:39:53.274830 25190 raft_consensus.cc:2258] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:53.275360 25184 raft_consensus.cc:2258] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:53.277698 25190 ts_tablet_manager.cc:1905] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:53.280033 25184 ts_tablet_manager.cc:1905] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:53.283210 25188 tablet_replica.cc:330] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: stopping tablet replica
I20260809 05:39:53.283705 25188 raft_consensus.cc:2229] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:39:53.284425 25188 raft_consensus.cc:2258] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:53.286932 25188 ts_tablet_manager.cc:1905] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:53.287501 25190 ts_tablet_manager.cc:1918] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:53.287686 25190 log.cc:1196] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/a5f118bf4c2048ccb05718f610b9fae0
I20260809 05:39:53.288296 25190 ts_tablet_manager.cc:1939] T a5f118bf4c2048ccb05718f610b9fae0 P 076b517be7f44f42ad12ff04c4e112d6: Deleting consensus metadata
I20260809 05:39:53.290289 24493 catalog_manager.cc:4841] TS 076b517be7f44f42ad12ff04c4e112d6 (127.30.137.194:46547): tablet a5f118bf4c2048ccb05718f610b9fae0 (table test-table [id=b0b860d347bb424ca418886e3cf65ac1]) successfully deleted
I20260809 05:39:53.292112 25184 ts_tablet_manager.cc:1918] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:53.292317 25184 log.cc:1196] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/a5f118bf4c2048ccb05718f610b9fae0
I20260809 05:39:53.292973 25184 ts_tablet_manager.cc:1939] T a5f118bf4c2048ccb05718f610b9fae0 P 76ff9345b7664a889c117a947a0a9ee2: Deleting consensus metadata
I20260809 05:39:53.294935 24494 catalog_manager.cc:4841] TS 76ff9345b7664a889c117a947a0a9ee2 (127.30.137.193:46859): tablet a5f118bf4c2048ccb05718f610b9fae0 (table test-table [id=b0b860d347bb424ca418886e3cf65ac1]) successfully deleted
I20260809 05:39:53.295361 25188 ts_tablet_manager.cc:1918] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:53.295629 25188 log.cc:1196] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/a5f118bf4c2048ccb05718f610b9fae0
I20260809 05:39:53.296198 25188 ts_tablet_manager.cc:1939] T a5f118bf4c2048ccb05718f610b9fae0 P ce0e5834fff4464b8d0650bf156ace12: Deleting consensus metadata
I20260809 05:39:53.297775 24495 catalog_manager.cc:4841] TS ce0e5834fff4464b8d0650bf156ace12 (127.30.137.195:40721): tablet a5f118bf4c2048ccb05718f610b9fae0 (table test-table [id=b0b860d347bb424ca418886e3cf65ac1]) successfully deleted
Aug 09 05:39:53 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:39:53.307142 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.269521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.30.137.193:46859)
0809 05:39:53.269758 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:53.269767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:53.288565 (+ 18798us) server_negotiation.cc:407] Connection header received
0809 05:39:53.289098 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:53.289105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:53.289188 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:53.289278 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:53.291271 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.292518 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.294666 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.295321 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.295489 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.300930 (+ 5441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:53.300944 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:53.300958 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:53.300992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:53.304067 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:53.304842 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:53.304851 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:53.304854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:53.304997 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:53.305430 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:53.305439 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:53.305441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:53.305824 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:53.306161 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:53.306489 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 05:39:53.306766 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 05:39:53.307984 24756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:53.400110 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 24595
I20260809 05:39:53.425482 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 24768
Aug 09 05:39:53 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:39:53.448729 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.426416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.30.137.194:46547)
0809 05:39:53.426613 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:53.426621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:53.426644 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:39:53.426953 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:53.426958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:53.427028 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:53.427146 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:53.428494 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.429243 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.430569 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.431215 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.431365 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.441277 (+ 9912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:53.441301 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:53.441324 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:53.441387 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:53.445353 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:53.446257 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:53.446268 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:53.446271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:53.446458 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:53.446935 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:53.446941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:53.446944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:53.447489 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:53.447809 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:53.448067 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 05:39:53.448318 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:39:53.449482 24939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:53.542724 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 24768
I20260809 05:39:53.572041 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 24953
Aug 09 05:39:53 dist-test-slave-3kk6 krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:39:53.590157 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:53.572953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.30.137.195:40721)
0809 05:39:53.573176 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:53.573183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:53.573207 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:39:53.573401 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:53.573407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:53.573515 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:53.573641 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:53.575290 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.576576 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.578022 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:53.578561 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:53.578860 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:53.583566 (+ 4706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:53.583580 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:53.583599 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:53.583635 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:53.587062 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:53.587831 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:53.587848 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:53.587851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:53.588023 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:53.588470 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:53.588475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:53.588478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:53.588860 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:53.589210 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:53.589533 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 05:39:53.589761 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:39:53.591028 25120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:53.717087 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 24953
I20260809 05:39:53.742275 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 24473
I20260809 05:39:53.743556 24552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:53.856638 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 24473
2026-08-09T05:39:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5174 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T05:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:39:53Z Disabled control of system clock
I20260809 05:39:53.966248 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:32987
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:32987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:39:54.337152 25202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:54.337494 25202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:54.337581 25202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:54.345305 25202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:39:54.345453 25202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:54.345500 25202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:54.345563 25202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:39:54.345602 25202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:39:54.355909 25202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:32987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:32987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:54.357332 25202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:54.359292 25202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:54.371928 25208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:54.371889 25209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:54.371955 25212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:54.373061 25202 server_base.cc:1034] running on GCE node
I20260809 05:39:54.374049 25202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:54.375797 25202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:54.377035 25202 hybrid_clock.cc:648] HybridClock initialized: now 1786253994376989 us; error 45 us; skew 500 ppm
I20260809 05:39:54.377436 25202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:39:54.380129 25202 webserver.cc:460] Webserver started at http://127.30.137.254:35339/ using document root <none> and password file <none>
I20260809 05:39:54.380905 25202 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:54.381008 25202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:54.381404 25202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:54.384620 25202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "e1cfdf61f4d047df9b1a4dac9da08921"
format_stamp: "Formatted at 2026-08-09 05:39:54 on dist-test-slave-3kk6"
server_key: "945b991e2dbfdc29dba8a78a956c151c"
server_key_iv: "9e9ae14a1f0f1b116afe2a53e83901b0"
server_key_version: "encryptionkey@0"
I20260809 05:39:54.385541 25202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "e1cfdf61f4d047df9b1a4dac9da08921"
format_stamp: "Formatted at 2026-08-09 05:39:54 on dist-test-slave-3kk6"
server_key: "945b991e2dbfdc29dba8a78a956c151c"
server_key_iv: "9e9ae14a1f0f1b116afe2a53e83901b0"
server_key_version: "encryptionkey@0"
I20260809 05:39:54.391321 25202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 05:39:54.395332 25220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:54.396895 25202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:39:54.397118 25202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "e1cfdf61f4d047df9b1a4dac9da08921"
format_stamp: "Formatted at 2026-08-09 05:39:54 on dist-test-slave-3kk6"
server_key: "945b991e2dbfdc29dba8a78a956c151c"
server_key_iv: "9e9ae14a1f0f1b116afe2a53e83901b0"
server_key_version: "encryptionkey@0"
I20260809 05:39:54.397357 25202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:54.413560 25202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:54.414513 25202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:54.414880 25202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:54.434713 25202 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:32987
I20260809 05:39:54.434791 25287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:32987 every 8 connection(s)
I20260809 05:39:54.436239 25202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:39:54.440327 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 25202
I20260809 05:39:54.440451 25288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:54.440532 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:39:54.441016 31271 external_mini_cluster.cc:1428] Setting key be71b3340795f603f1828da0bf463f36
I20260809 05:39:54.454351 25288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921: Bootstrap starting.
I20260809 05:39:54.455540 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:54.443113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.30.137.254:32987)
0809 05:39:54.444255 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:54.444266 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:54.444314 (+ 48us) server_negotiation.cc:407] Connection header received
0809 05:39:54.446023 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:54.446048 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:54.446542 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:54.447055 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:54.447696 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:54.449276 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:54.450781 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:54.451517 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:54.451773 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:54.453076 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:54.453084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:54.453099 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:54.453657 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:54.453741 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:54.453761 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:54.453912 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:54.454141 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 05:39:54.454504 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":410,"threads_started":1}
I20260809 05:39:54.457840 25288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:54.459133 25288 log.cc:824] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:54.461971 25288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921: No bootstrap required, opened a new log
I20260809 05:39:54.470469 25288 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 32987 } }
I20260809 05:39:54.471009 25288 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:54.471124 25288 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1cfdf61f4d047df9b1a4dac9da08921, State: Initialized, Role: FOLLOWER
I20260809 05:39:54.471773 25288 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 32987 } }
I20260809 05:39:54.472048 25288 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:39:54.472193 25288 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:39:54.472426 25288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:54.475339 25288 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 32987 } }
I20260809 05:39:54.475919 25288 leader_election.cc:302] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1cfdf61f4d047df9b1a4dac9da08921; no voters:
I20260809 05:39:54.476572 25288 leader_election.cc:288] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:39:54.476742 25296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:39:54.477566 25296 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [term 1 LEADER]: Becoming Leader. State: Replica: e1cfdf61f4d047df9b1a4dac9da08921, State: Running, Role: LEADER
I20260809 05:39:54.478154 25296 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 32987 } }
I20260809 05:39:54.478921 25288 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:39:54.481424 25298 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 32987 } } }
I20260809 05:39:54.481665 25299 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1cfdf61f4d047df9b1a4dac9da08921. Latest consensus state: current_term: 1 leader_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1cfdf61f4d047df9b1a4dac9da08921" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 32987 } } }
I20260809 05:39:54.481953 25298 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:54.481976 25299 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921 [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:54.483448 25308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:39:54.488411 25308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:39:54.498724 25308 catalog_manager.cc:1347] Generated new cluster ID: 39d4a78289e143e5b9f51179df35616a
I20260809 05:39:54.498909 25308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:39:54.510601 25308 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:39:54.512187 25308 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:39:54.543629 25308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921: Generated new TSK 0
I20260809 05:39:54.544595 25308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:39:54.557638 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:32987
--builtin_ntp_servers=127.30.137.212:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:39:54.933759 25322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:54.934096 25322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:54.934194 25322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:54.941716 25322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:54.941861 25322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:54.941974 25322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:39:54.951474 25322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:32987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:54.952930 25322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:54.954916 25322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:54.969938 25333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:54.969913 25329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:54.970201 25330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:54.970922 25322 server_base.cc:1034] running on GCE node
I20260809 05:39:54.971627 25322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:54.972747 25322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:54.974025 25322 hybrid_clock.cc:648] HybridClock initialized: now 1786253994973990 us; error 66 us; skew 500 ppm
I20260809 05:39:54.974397 25322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:39:54.977340 25322 webserver.cc:460] Webserver started at http://127.30.137.193:33015/ using document root <none> and password file <none>
I20260809 05:39:54.978148 25322 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:54.978318 25322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:54.978776 25322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:54.982053 25322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "2fe38258005146d59927d763c6b5412f"
format_stamp: "Formatted at 2026-08-09 05:39:54 on dist-test-slave-3kk6"
server_key: "73d32e3815c72ed37884cf5c357a2264"
server_key_iv: "5d2020b131eb93b49a4b22c597e25892"
server_key_version: "encryptionkey@0"
I20260809 05:39:54.983244 25322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "2fe38258005146d59927d763c6b5412f"
format_stamp: "Formatted at 2026-08-09 05:39:54 on dist-test-slave-3kk6"
server_key: "73d32e3815c72ed37884cf5c357a2264"
server_key_iv: "5d2020b131eb93b49a4b22c597e25892"
server_key_version: "encryptionkey@0"
I20260809 05:39:54.989076 25322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:39:54.992923 25339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:54.994333 25322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 05:39:54.994540 25322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "2fe38258005146d59927d763c6b5412f"
format_stamp: "Formatted at 2026-08-09 05:39:54 on dist-test-slave-3kk6"
server_key: "73d32e3815c72ed37884cf5c357a2264"
server_key_iv: "5d2020b131eb93b49a4b22c597e25892"
server_key_version: "encryptionkey@0"
I20260809 05:39:54.994868 25322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:55.022212 25322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:55.023212 25322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:55.023573 25322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:55.024680 25322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:55.026298 25322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:55.026435 25322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:55.026589 25322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:55.026705 25322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:55.062389 25322 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:43003
I20260809 05:39:55.062491 25496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:43003 every 8 connection(s)
I20260809 05:39:55.063859 25322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:39:55.069010 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 25322
I20260809 05:39:55.069214 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:39:55.069693 31271 external_mini_cluster.cc:1428] Setting key 59f904123fed04f952aee5761f50084e
I20260809 05:39:55.072638 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:32987
--builtin_ntp_servers=127.30.137.212:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:39:55.077442 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:55.066546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:35205 (local address 127.30.137.254:32987)
0809 05:39:55.067084 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:55.067088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:55.068302 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 05:39:55.070402 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:55.070409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:55.070507 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:55.070646 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:55.071835 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.072570 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.074304 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.074806 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.074965 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:55.076507 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:55.076512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:55.076515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:55.076536 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:55.076579 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:55.076591 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:55.076695 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:55.076955 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 05:39:55.077068 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":276,"threads_started":1}
I20260809 05:39:55.078545 25504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:55.067201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:32987 (local address 127.30.137.193:35205)
0809 05:39:55.068127 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:55.068165 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:55.070084 (+ 1919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:55.070935 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:55.070943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:55.071037 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:55.071632 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:55.071668 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.072755 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.072759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:55.074131 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:55.074146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.075072 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.075079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:55.075334 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:55.076144 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:39:55.076147 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:55.076196 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:39:55.076283 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:55.076696 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:55.076710 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:55.076721 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 05:39:55.076931 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 05:39:55.077357 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":278,"threads_started":1}
I20260809 05:39:55.080518 25498 heartbeater.cc:347] Connected to a master server at 127.30.137.254:32987
I20260809 05:39:55.080842 25498 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:55.081620 25498 heartbeater.cc:511] Master 127.30.137.254:32987 requested a full tablet report, sending...
I20260809 05:39:55.084184 25244 ts_manager.cc:194] Registered new tserver with Master: 2fe38258005146d59927d763c6b5412f (127.30.137.193:43003)
I20260809 05:39:55.086426 25244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.193:35205
W20260809 05:39:55.437518 25505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:55.437894 25505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:55.438069 25505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:55.445689 25505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:55.445879 25505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:55.446069 25505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:39:55.455900 25505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:32987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:55.457432 25505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:55.459498 25505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:55.474522 25517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:55.474563 25514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:55.474614 25513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:55.481045 25505 server_base.cc:1034] running on GCE node
I20260809 05:39:55.481765 25505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:55.483722 25505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:55.484987 25505 hybrid_clock.cc:648] HybridClock initialized: now 1786253995484893 us; error 95 us; skew 500 ppm
I20260809 05:39:55.485370 25505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:39:55.488050 25505 webserver.cc:460] Webserver started at http://127.30.137.194:36795/ using document root <none> and password file <none>
I20260809 05:39:55.488929 25505 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:55.489076 25505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:55.489445 25505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:55.492884 25505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "359d4ec8cb734d6a8dfc681410b32251"
format_stamp: "Formatted at 2026-08-09 05:39:55 on dist-test-slave-3kk6"
server_key: "f624e7649471915eba164916dfab1270"
server_key_iv: "05b98ae187cdbf76f1ce485553b4c51f"
server_key_version: "encryptionkey@0"
I20260809 05:39:55.493800 25505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "359d4ec8cb734d6a8dfc681410b32251"
format_stamp: "Formatted at 2026-08-09 05:39:55 on dist-test-slave-3kk6"
server_key: "f624e7649471915eba164916dfab1270"
server_key_iv: "05b98ae187cdbf76f1ce485553b4c51f"
server_key_version: "encryptionkey@0"
I20260809 05:39:55.500037 25505 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 05:39:55.504367 25530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:55.505951 25505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260809 05:39:55.506170 25505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "359d4ec8cb734d6a8dfc681410b32251"
format_stamp: "Formatted at 2026-08-09 05:39:55 on dist-test-slave-3kk6"
server_key: "f624e7649471915eba164916dfab1270"
server_key_iv: "05b98ae187cdbf76f1ce485553b4c51f"
server_key_version: "encryptionkey@0"
I20260809 05:39:55.506431 25505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:55.521113 25505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:55.522112 25505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:55.522490 25505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:55.523721 25505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:55.525413 25505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:55.525544 25505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:55.525701 25505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:55.525817 25505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:55.563122 25505 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:37305
I20260809 05:39:55.563211 25679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:37305 every 8 connection(s)
I20260809 05:39:55.564633 25505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:39:55.570216 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 25505
I20260809 05:39:55.570472 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:39:55.571101 31271 external_mini_cluster.cc:1428] Setting key dc0ecd4ebe5bbb74903c633cf581385a
I20260809 05:39:55.574364 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:32987
--builtin_ntp_servers=127.30.137.212:44971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:39:55.579939 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:55.567485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:56025 (local address 127.30.137.254:32987)
0809 05:39:55.567738 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:55.567745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:55.569494 (+ 1749us) server_negotiation.cc:407] Connection header received
0809 05:39:55.572275 (+ 2781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:55.572280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:55.572358 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:55.572496 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:55.573897 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.574869 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.576584 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.576999 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.577150 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:55.578771 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:55.578783 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:55.578787 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:55.578810 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:55.578845 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:55.578859 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:55.578969 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:55.579307 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 05:39:55.579474 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 05:39:55.581072 25686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:55.568109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:32987 (local address 127.30.137.194:56025)
0809 05:39:55.569285 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:55.569337 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:55.571899 (+ 2562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:55.572791 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:55.572800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:55.572920 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:55.573700 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:55.573723 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.575043 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.575053 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:55.576405 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:55.576420 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:55.577255 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:55.577263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:55.577521 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:55.578340 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:39:55.578343 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:55.578393 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:39:55.578486 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:55.578984 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:55.578998 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:55.579007 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 05:39:55.579319 (+ 312us) client_negotiation.cc:240] Negotiation successful
0809 05:39:55.579732 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":398,"threads_started":1}
I20260809 05:39:55.582901 25680 heartbeater.cc:347] Connected to a master server at 127.30.137.254:32987
I20260809 05:39:55.583325 25680 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:55.584359 25680 heartbeater.cc:511] Master 127.30.137.254:32987 requested a full tablet report, sending...
I20260809 05:39:55.586169 25244 ts_manager.cc:194] Registered new tserver with Master: 359d4ec8cb734d6a8dfc681410b32251 (127.30.137.194:37305)
I20260809 05:39:55.587103 25244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.194:56025
W20260809 05:39:55.941237 25687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:55.941645 25687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:55.941817 25687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:55.949510 25687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:55.949697 25687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:55.949901 25687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:39:55.959671 25687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:44971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:32987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:55.961181 25687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:55.963263 25687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:55.977639 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:55.977602 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:55.977545 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:55.978564 25687 server_base.cc:1034] running on GCE node
I20260809 05:39:55.979318 25687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:55.980417 25687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:55.981714 25687 hybrid_clock.cc:648] HybridClock initialized: now 1786253995981656 us; error 68 us; skew 500 ppm
I20260809 05:39:55.982206 25687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:39:55.984954 25687 webserver.cc:460] Webserver started at http://127.30.137.195:40605/ using document root <none> and password file <none>
I20260809 05:39:55.985728 25687 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:55.985873 25687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:55.986212 25687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:55.989413 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "bab0a572843b453cae891be86501d1d9"
format_stamp: "Formatted at 2026-08-09 05:39:55 on dist-test-slave-3kk6"
server_key: "32f6d9e0056cca0b3676f70082c12d3e"
server_key_iv: "f8883e8938cbea4fff72168f7029b395"
server_key_version: "encryptionkey@0"
I20260809 05:39:55.990309 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "bab0a572843b453cae891be86501d1d9"
format_stamp: "Formatted at 2026-08-09 05:39:55 on dist-test-slave-3kk6"
server_key: "32f6d9e0056cca0b3676f70082c12d3e"
server_key_iv: "f8883e8938cbea4fff72168f7029b395"
server_key_version: "encryptionkey@0"
I20260809 05:39:55.996227 25687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 05:39:56.000059 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:56.001612 25687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:39:56.001767 25687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "bab0a572843b453cae891be86501d1d9"
format_stamp: "Formatted at 2026-08-09 05:39:55 on dist-test-slave-3kk6"
server_key: "32f6d9e0056cca0b3676f70082c12d3e"
server_key_iv: "f8883e8938cbea4fff72168f7029b395"
server_key_version: "encryptionkey@0"
I20260809 05:39:56.001963 25687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:56.023146 25687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:56.023996 25687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:56.024384 25687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:56.025424 25687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:39:56.026978 25687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:39:56.027072 25687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:56.027154 25687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:39:56.027210 25687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:56.062665 25687 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:36583
I20260809 05:39:56.062736 25861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:36583 every 8 connection(s)
I20260809 05:39:56.063988 25687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:39:56.071326 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 25687
I20260809 05:39:56.071569 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:39:56.072206 31271 external_mini_cluster.cc:1428] Setting key 18dcf3ca2f46e0211c5cdd2aa8eb0714
I20260809 05:39:56.077920 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.066824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:60651 (local address 127.30.137.254:32987)
0809 05:39:56.067057 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.067065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.068424 (+ 1359us) server_negotiation.cc:407] Connection header received
0809 05:39:56.070315 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.070321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.070429 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.070548 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:56.072152 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.073203 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.075202 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.075646 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.075748 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.077189 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:56.077193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:56.077196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:56.077215 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:56.077245 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:56.077256 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:56.077320 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.077564 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.077649 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 05:39:56.079110 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.067335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:32987 (local address 127.30.137.195:60651)
0809 05:39:56.068248 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.068284 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:56.070014 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:56.070815 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:56.070824 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:56.070988 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:39:56.071915 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.071946 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.073403 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.073407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.075052 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.075066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.075862 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.075868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.076075 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.076868 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:39:56.076871 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:39:56.076912 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:39:56.077001 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:39:56.077349 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:39:56.077358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:39:56.077364 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:39:56.077520 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 05:39:56.077874 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":283,"threads_started":1}
I20260809 05:39:56.080716 25862 heartbeater.cc:347] Connected to a master server at 127.30.137.254:32987
I20260809 05:39:56.081066 25862 heartbeater.cc:464] Registering TS with master...
I20260809 05:39:56.081975 25862 heartbeater.cc:511] Master 127.30.137.254:32987 requested a full tablet report, sending...
I20260809 05:39:56.083671 25244 ts_manager.cc:194] Registered new tserver with Master: bab0a572843b453cae891be86501d1d9 (127.30.137.195:36583)
I20260809 05:39:56.084501 25244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.195:60651
I20260809 05:39:56.087899 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:39:56.090512 25498 heartbeater.cc:503] Master 127.30.137.254:32987 was elected leader, sending a full tablet report...
I20260809 05:39:56.100432 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.092625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.30.137.254:32987)
0809 05:39:56.092814 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.092818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.093295 (+ 477us) server_negotiation.cc:407] Connection header received
0809 05:39:56.093435 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.093438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.093498 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.093578 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:56.094080 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.094879 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.097306 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.097730 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.097826 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.099470 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:56.099478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:56.099484 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:56.099513 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:56.099557 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:56.099574 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:56.099692 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.099935 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.100051 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:39:56.106223 25244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:39:56.115468 25244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:39:56.143620 25894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.128262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:36583 (local address 127.0.0.1:53052)
0809 05:39:56.130057 (+ 1795us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.130076 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:56.130192 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:56.134942 (+ 4750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:56.134950 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:56.134984 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:56.135483 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.135498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.137743 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.137748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.140522 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.140541 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.142457 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.142463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.142754 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.142820 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 05:39:56.142967 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 05:39:56.143185 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1658,"mutex_wait_us":42,"thread_start_us":426,"threads_started":1}
I20260809 05:39:56.143941 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.128073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37305 (local address 127.0.0.1:35692)
0809 05:39:56.129027 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.129065 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:56.129254 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:56.135971 (+ 6717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:56.135976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:56.136004 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:56.136363 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.136374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.138250 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.138255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.140288 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.140306 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.142435 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.142445 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.143222 (+ 777us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.143253 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:39:56.143342 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 05:39:56.143468 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":568,"thread_start_us":6695,"threads_started":1}
I20260809 05:39:56.144476 25892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.128996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53052 (local address 127.30.137.195:36583)
0809 05:39:56.134099 (+ 5103us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.134108 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.134148 (+ 40us) server_negotiation.cc:407] Connection header received
0809 05:39:56.134267 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.134276 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.134596 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.135680 (+ 1084us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:56.135999 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.137578 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.140750 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.142164 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.143313 (+ 1149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.143533 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.143611 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.143996 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4914,"thread_start_us":4737,"threads_started":1}
I20260809 05:39:56.144563 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.130133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.30.137.193:43003)
0809 05:39:56.130771 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.130781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.135370 (+ 4589us) server_negotiation.cc:407] Connection header received
0809 05:39:56.135486 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.135493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.135772 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.135987 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:56.136900 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.138414 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.141168 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.142643 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.142911 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.143134 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.143919 (+ 785us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.144194 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":873,"threads_started":1}
I20260809 05:39:56.144766 25891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.128295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.30.137.194:37305)
0809 05:39:56.135460 (+ 7165us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.135467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.135494 (+ 27us) server_negotiation.cc:407] Connection header received
0809 05:39:56.135580 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.135586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.135770 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.135980 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:56.136558 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.138086 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.140548 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.142164 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.142777 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.143026 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.144257 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.144410 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7015,"thread_start_us":6887,"threads_started":1}
I20260809 05:39:56.144906 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.129503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:36992)
0809 05:39:56.135074 (+ 5571us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.135097 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:56.135230 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:56.136254 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:56.136259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:56.136291 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:56.136713 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.136725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.138568 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.138574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.140969 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.140980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.142922 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.142929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.143108 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.143173 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 05:39:56.143270 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 05:39:56.143410 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5439,"mutex_wait_us":40,"thread_start_us":829,"threads_started":1}
I20260809 05:39:56.148602 25773 tablet_service.cc:1467] Processing CreateTablet for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 (DEFAULT_TABLE table=test-table [id=07cc2a6ae68c4df7b4a15210b2913fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:56.148675 25590 tablet_service.cc:1467] Processing CreateTablet for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 (DEFAULT_TABLE table=test-table [id=07cc2a6ae68c4df7b4a15210b2913fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:56.148960 25408 tablet_service.cc:1467] Processing CreateTablet for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 (DEFAULT_TABLE table=test-table [id=07cc2a6ae68c4df7b4a15210b2913fdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:39:56.150044 25773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:56.150130 25590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:56.150236 25408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:56.162937 25896 tablet_bootstrap.cc:492] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Bootstrap starting.
I20260809 05:39:56.165411 25898 tablet_bootstrap.cc:492] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Bootstrap starting.
I20260809 05:39:56.167491 25897 tablet_bootstrap.cc:492] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Bootstrap starting.
I20260809 05:39:56.167534 25896 tablet_bootstrap.cc:654] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:56.169042 25896 log.cc:824] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:56.169296 25898 tablet_bootstrap.cc:654] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:56.170596 25898 log.cc:824] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:56.172173 25897 tablet_bootstrap.cc:654] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:56.173511 25897 log.cc:824] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Log is configured to *not* fsync() on all Append() calls
I20260809 05:39:56.177639 25898 tablet_bootstrap.cc:492] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: No bootstrap required, opened a new log
I20260809 05:39:56.177999 25897 tablet_bootstrap.cc:492] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: No bootstrap required, opened a new log
I20260809 05:39:56.177983 25898 ts_tablet_manager.cc:1397] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260809 05:39:56.178274 25897 ts_tablet_manager.cc:1397] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 05:39:56.178359 25896 tablet_bootstrap.cc:492] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: No bootstrap required, opened a new log
I20260809 05:39:56.178675 25896 ts_tablet_manager.cc:1397] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.000s
I20260809 05:39:56.186746 25897 raft_consensus.cc:348] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.187211 25897 raft_consensus.cc:374] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:56.187316 25897 raft_consensus.cc:729] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fe38258005146d59927d763c6b5412f, State: Initialized, Role: FOLLOWER
I20260809 05:39:56.188005 25897 consensus_queue.cc:260] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.189523 25898 raft_consensus.cc:348] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.189541 25896 raft_consensus.cc:348] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.190012 25898 raft_consensus.cc:374] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:56.190012 25896 raft_consensus.cc:374] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:56.190053 25897 ts_tablet_manager.cc:1428] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260809 05:39:56.190119 25896 raft_consensus.cc:729] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab0a572843b453cae891be86501d1d9, State: Initialized, Role: FOLLOWER
I20260809 05:39:56.190110 25898 raft_consensus.cc:729] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359d4ec8cb734d6a8dfc681410b32251, State: Initialized, Role: FOLLOWER
I20260809 05:39:56.190789 25896 consensus_queue.cc:260] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.190807 25898 consensus_queue.cc:260] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.192483 25862 heartbeater.cc:503] Master 127.30.137.254:32987 was elected leader, sending a full tablet report...
I20260809 05:39:56.192785 25898 ts_tablet_manager.cc:1428] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 05:39:56.193071 25896 ts_tablet_manager.cc:1428] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 05:39:56.194016 25680 heartbeater.cc:503] Master 127.30.137.254:32987 was elected leader, sending a full tablet report...
W20260809 05:39:56.315912 25863 tablet.cc:2369] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:39:56.316524 25499 tablet.cc:2369] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:39:56.316921 25681 tablet.cc:2369] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:39:56.535590 25905 raft_consensus.cc:482] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:39:56.536046 25905 raft_consensus.cc:504] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.538138 25905 leader_election.cc:288] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 359d4ec8cb734d6a8dfc681410b32251 (127.30.137.194:37305), bab0a572843b453cae891be86501d1d9 (127.30.137.195:36583)
I20260809 05:39:56.546033 25892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.538704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:58587 (local address 127.30.137.195:36583)
0809 05:39:56.538949 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.538955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.540057 (+ 1102us) server_negotiation.cc:407] Connection header received
0809 05:39:56.540257 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.540260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.540321 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.540449 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:56.541151 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.541863 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.544182 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.545014 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.545190 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.545280 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.545627 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.545767 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 05:39:56.546053 25891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.538759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:60949 (local address 127.30.137.194:37305)
0809 05:39:56.538992 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.539000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.539966 (+ 966us) server_negotiation.cc:407] Connection header received
0809 05:39:56.540206 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.540214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.540297 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.540374 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:39:56.541087 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.541852 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.544270 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.545055 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.545257 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.545327 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.545631 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.545768 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:39:56.546396 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.538734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:37305 (local address 127.30.137.193:60949)
0809 05:39:56.539798 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.539817 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:56.540015 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:56.540448 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:56.540453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:56.540478 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:56.540917 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.540928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.542003 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.542010 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.543951 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.543965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.545310 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.545317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.545448 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.545476 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 05:39:56.545595 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 05:39:56.545754 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":105,"thread_start_us":594,"threads_started":1}
I20260809 05:39:56.546424 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.538837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:36583 (local address 127.30.137.193:58587)
0809 05:39:56.539877 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.539893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:39:56.540105 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:39:56.540490 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:39:56.540495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:39:56.540523 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:39:56.540951 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.540960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.542051 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.542055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.543945 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:39:56.543963 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.545248 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.545254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:39:56.545446 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.545472 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 05:39:56.545598 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 05:39:56.545755 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":110,"thread_start_us":669,"threads_started":1}
I20260809 05:39:56.547632 25797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ee2f2cdfa949b5a4e6aff312e36ee9" candidate_uuid: "2fe38258005146d59927d763c6b5412f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab0a572843b453cae891be86501d1d9" is_pre_election: true
I20260809 05:39:56.547633 25614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ee2f2cdfa949b5a4e6aff312e36ee9" candidate_uuid: "2fe38258005146d59927d763c6b5412f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359d4ec8cb734d6a8dfc681410b32251" is_pre_election: true
I20260809 05:39:56.548273 25797 raft_consensus.cc:2454] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fe38258005146d59927d763c6b5412f in term 0.
I20260809 05:39:56.548273 25614 raft_consensus.cc:2454] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fe38258005146d59927d763c6b5412f in term 0.
I20260809 05:39:56.549062 25344 leader_election.cc:302] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fe38258005146d59927d763c6b5412f, bab0a572843b453cae891be86501d1d9; no voters:
I20260809 05:39:56.549592 25905 raft_consensus.cc:2781] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:39:56.549774 25905 raft_consensus.cc:482] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:39:56.549916 25905 raft_consensus.cc:3037] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:56.552323 25905 raft_consensus.cc:504] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.553009 25905 leader_election.cc:288] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [CANDIDATE]: Term 1 election: Requested vote from peers 359d4ec8cb734d6a8dfc681410b32251 (127.30.137.194:37305), bab0a572843b453cae891be86501d1d9 (127.30.137.195:36583)
I20260809 05:39:56.553591 25614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ee2f2cdfa949b5a4e6aff312e36ee9" candidate_uuid: "2fe38258005146d59927d763c6b5412f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359d4ec8cb734d6a8dfc681410b32251"
I20260809 05:39:56.553591 25797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ee2f2cdfa949b5a4e6aff312e36ee9" candidate_uuid: "2fe38258005146d59927d763c6b5412f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bab0a572843b453cae891be86501d1d9"
I20260809 05:39:56.553819 25614 raft_consensus.cc:3037] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:56.554054 25797 raft_consensus.cc:3037] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:56.556311 25614 raft_consensus.cc:2454] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fe38258005146d59927d763c6b5412f in term 1.
I20260809 05:39:56.556607 25797 raft_consensus.cc:2454] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fe38258005146d59927d763c6b5412f in term 1.
I20260809 05:39:56.556972 25346 leader_election.cc:302] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fe38258005146d59927d763c6b5412f, 359d4ec8cb734d6a8dfc681410b32251; no voters:
I20260809 05:39:56.557330 25905 raft_consensus.cc:2781] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:39:56.557792 25905 raft_consensus.cc:686] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 1 LEADER]: Becoming Leader. State: Replica: 2fe38258005146d59927d763c6b5412f, State: Running, Role: LEADER
I20260809 05:39:56.558306 25905 consensus_queue.cc:237] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } }
I20260809 05:39:56.563657 25239 catalog_manager.cc:5495] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fe38258005146d59927d763c6b5412f (127.30.137.193). New cstate: current_term: 1 leader_uuid: "2fe38258005146d59927d763c6b5412f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fe38258005146d59927d763c6b5412f" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 43003 } health_report { overall_health: HEALTHY } } }
I20260809 05:39:56.604022 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.596434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.30.137.193:43003)
0809 05:39:56.596614 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.596621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.596746 (+ 125us) server_negotiation.cc:407] Connection header received
0809 05:39:56.597022 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.597028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.597165 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.597337 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:39:56.597953 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.599202 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.601470 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.602011 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.602168 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.602869 (+ 701us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:39:56.603417 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:39:56.603596 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.603650 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.603748 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:39:56.613987 25614 raft_consensus.cc:1261] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 1 FOLLOWER]: Refusing update from remote peer 2fe38258005146d59927d763c6b5412f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:39:56.613986 25797 raft_consensus.cc:1261] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 1 FOLLOWER]: Refusing update from remote peer 2fe38258005146d59927d763c6b5412f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:39:56.615100 25905 consensus_queue.cc:1035] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bab0a572843b453cae891be86501d1d9" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 36583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:39:56.615437 25911 consensus_queue.cc:1035] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [LEADER]: Connected to new peer: Peer: permanent_uuid: "359d4ec8cb734d6a8dfc681410b32251" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 37305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:39:56.628860 25917 mvcc.cc:205] Tried to move back new op lower bound from 7316496370120482816 to 7316496369906925568. Current Snapshot: MvccSnapshot[applied={T|T < 7316496370120482816}]
I20260809 05:39:56.629098 25919 mvcc.cc:205] Tried to move back new op lower bound from 7316496370120482816 to 7316496369906925568. Current Snapshot: MvccSnapshot[applied={T|T < 7316496370120482816}]
I20260809 05:39:56.648010 25244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:39:56.648367 25244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:39:56.652598 25244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1cfdf61f4d047df9b1a4dac9da08921: Sending DeleteTablet for 3 replicas of tablet 47ee2f2cdfa949b5a4e6aff312e36ee9
I20260809 05:39:56.654014 25590 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:56 UTC) from {username='slave'} at 127.0.0.1:35692
I20260809 05:39:56.654014 25773 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:56 UTC) from {username='slave'} at 127.0.0.1:53052
I20260809 05:39:56.654270 25408 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:39:56 UTC) from {username='slave'} at 127.0.0.1:36992
I20260809 05:39:56.656968 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 25322
I20260809 05:39:56.664927 25937 tablet_replica.cc:330] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: stopping tablet replica
I20260809 05:39:56.665598 25937 raft_consensus.cc:2229] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:39:56.666368 25937 raft_consensus.cc:2258] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:56.668903 25938 tablet_replica.cc:330] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: stopping tablet replica
I20260809 05:39:56.669533 25938 raft_consensus.cc:2229] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:39:56.670473 25938 raft_consensus.cc:2258] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:56.674152 25937 ts_tablet_manager.cc:1905] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:56.674175 25938 ts_tablet_manager.cc:1905] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:56.679273 25936 tablet_replica.cc:330] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: stopping tablet replica
I20260809 05:39:56.682305 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.658821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.30.137.193:43003)
0809 05:39:56.659056 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.659064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.671393 (+ 12329us) server_negotiation.cc:407] Connection header received
0809 05:39:56.671506 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.671512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.671638 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.671810 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:56.672659 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.674028 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.676499 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.677339 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.677471 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.680082 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:56.680096 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:56.680115 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:56.681168 (+ 1053us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:56.681252 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:56.681283 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:56.681391 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.681720 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.681853 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 05:39:56.683611 25938 ts_tablet_manager.cc:1918] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:56.684108 25481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:56.684140 25937 ts_tablet_manager.cc:1918] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:56.684332 25937 log.cc:1196] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/47ee2f2cdfa949b5a4e6aff312e36ee9
I20260809 05:39:56.684474 25938 log.cc:1196] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 2fe38258005146d59927d763c6b5412f: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/47ee2f2cdfa949b5a4e6aff312e36ee9
I20260809 05:39:56.684765 25936 raft_consensus.cc:2229] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:39:56.685070 25937 ts_tablet_manager.cc:1939] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P 359d4ec8cb734d6a8dfc681410b32251: Deleting consensus metadata
I20260809 05:39:56.685586 25936 raft_consensus.cc:2258] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:39:56.687717 25226 catalog_manager.cc:4841] TS 359d4ec8cb734d6a8dfc681410b32251 (127.30.137.194:37305): tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 (table test-table [id=07cc2a6ae68c4df7b4a15210b2913fdb]) successfully deleted
I20260809 05:39:56.688697 25936 ts_tablet_manager.cc:1905] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:39:56.695997 25936 ts_tablet_manager.cc:1918] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:39:56.696247 25936 log.cc:1196] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/47ee2f2cdfa949b5a4e6aff312e36ee9
I20260809 05:39:56.696775 25936 ts_tablet_manager.cc:1939] T 47ee2f2cdfa949b5a4e6aff312e36ee9 P bab0a572843b453cae891be86501d1d9: Deleting consensus metadata
I20260809 05:39:56.698709 25222 catalog_manager.cc:4841] TS bab0a572843b453cae891be86501d1d9 (127.30.137.195:36583): tablet 47ee2f2cdfa949b5a4e6aff312e36ee9 (table test-table [id=07cc2a6ae68c4df7b4a15210b2913fdb]) successfully deleted
I20260809 05:39:56.787202 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 25322
I20260809 05:39:56.812340 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 25505
W20260809 05:39:56.812273 25222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.137.193:43003 (error 108)
I20260809 05:39:56.814769 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.813887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37024)
0809 05:39:56.814304 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.814496 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":211,"threads_started":1}
W20260809 05:39:56.815097 25222 catalog_manager.cc:4584] TS 2fe38258005146d59927d763c6b5412f (127.30.137.193:43003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47ee2f2cdfa949b5a4e6aff312e36ee9: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
I20260809 05:39:56.819875 25891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.813356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.30.137.194:37305)
0809 05:39:56.813545 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.813549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.813569 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:39:56.813692 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.813697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.813780 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.813871 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:56.814451 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.815454 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.816809 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.817226 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.817320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.818394 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:56.818403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:56.818417 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:56.819212 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:56.819260 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:56.819280 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:56.819381 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.819520 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.819611 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:39:56.820586 25674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:56.868973 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.868357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37030)
0809 05:39:56.868652 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.868772 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260809 05:39:56.870249 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.869774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37046)
0809 05:39:56.869980 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.870072 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 05:39:56.913319 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 25505
I20260809 05:39:56.931150 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.930458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37050)
0809 05:39:56.930780 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.930924 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 05:39:56.932538 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.931968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37064)
0809 05:39:56.932202 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:56.932319 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 05:39:56.937839 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 25687
I20260809 05:39:56.945024 25892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:56.938763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.30.137.195:36583)
0809 05:39:56.938984 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:56.938990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:56.939014 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:39:56.939120 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:56.939125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:56.939242 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:56.939362 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:56.939787 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.940540 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.941879 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:56.942292 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:56.942443 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:56.943601 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:56.943611 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:56.943626 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:39:56.944202 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:56.944256 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:39:56.944276 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:56.944427 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:56.944553 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 05:39:56.944686 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:39:56.945736 25856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:57.013504 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:57.012951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37068)
0809 05:39:57.013202 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:57.013332 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 05:39:57.014803 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:57.014293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:43003 (local address 127.0.0.1:37074)
0809 05:39:57.014518 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 05:39:57.014596 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:43003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 05:39:57.037755 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 25687
I20260809 05:39:57.061076 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 25202
I20260809 05:39:57.062362 25283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:39:57.175663 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 25202
2026-08-09T05:39:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3307 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:39:57 dist-test-slave-3kk6 krb5kdc[25952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:39:57 dist-test-slave-3kk6 krb5kdc[25952](info): set up 2 sockets
Aug 09 05:39:57 dist-test-slave-3kk6 krb5kdc[25952](info): commencing operation
krb5kdc: starting...
W20260809 05:39:58.534233 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:39:58 dist-test-slave-3kk6 krb5kdc[25952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:39:58.877476 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:45011
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:45011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:59.235292 25970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:59.235612 25970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:59.235795 25970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:59.243110 25970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:39:59.243235 25970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:59.243343 25970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:59.243458 25970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:39:59.243544 25970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:39:59.253746 25970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:45011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:45011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:59.255196 25970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:59.256636 25970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:59.269403 25975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:59.269450 25976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:59.269943 25978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:59.270735 25970 server_base.cc:1034] running on GCE node
I20260809 05:39:59.271638 25970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:59.273137 25970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:59.274399 25970 hybrid_clock.cc:648] HybridClock initialized: now 1786253999274362 us; error 74 us; skew 500 ppm
I20260809 05:39:59.274852 25970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:59 dist-test-slave-3kk6 krb5kdc[25952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:59.282019 25970 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:39:59.283757 25970 webserver.cc:460] Webserver started at http://127.30.137.254:34557/ using document root <none> and password file <none>
I20260809 05:39:59.284482 25970 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:59.284595 25970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:59.284886 25970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:59.288166 25970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "40c39e75c45c4f9ca20d2ddaa240047b"
format_stamp: "Formatted at 2026-08-09 05:39:59 on dist-test-slave-3kk6"
server_key: "844b16403c67d8d3554965438d2e1317"
server_key_iv: "37ddf318cff1375e97cef7fa91faf3e3"
server_key_version: "encryptionkey@0"
I20260809 05:39:59.288969 25970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "40c39e75c45c4f9ca20d2ddaa240047b"
format_stamp: "Formatted at 2026-08-09 05:39:59 on dist-test-slave-3kk6"
server_key: "844b16403c67d8d3554965438d2e1317"
server_key_iv: "37ddf318cff1375e97cef7fa91faf3e3"
server_key_version: "encryptionkey@0"
I20260809 05:39:59.294590 25970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:39:59.298296 25986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:59.299849 25970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:39:59.300019 25970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "40c39e75c45c4f9ca20d2ddaa240047b"
format_stamp: "Formatted at 2026-08-09 05:39:59 on dist-test-slave-3kk6"
server_key: "844b16403c67d8d3554965438d2e1317"
server_key_iv: "37ddf318cff1375e97cef7fa91faf3e3"
server_key_version: "encryptionkey@0"
I20260809 05:39:59.300287 25970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:39:59.329945 25970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:39:59.334631 25970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:39:59.335003 25970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:39:59.355387 25970 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:45011
I20260809 05:39:59.355420 26060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:45011 every 8 connection(s)
I20260809 05:39:59.356897 25970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:39:59.360818 26061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:39:59.363087 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 25970
I20260809 05:39:59.363293 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:39:59.363876 31271 external_mini_cluster.cc:1428] Setting key ae613c6a164df2f97f634f69a704393d
I20260809 05:39:59.374239 26061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b: Bootstrap starting.
I20260809 05:39:59.378574 26061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b: Neither blocks nor log segments found. Creating new log.
I20260809 05:39:59.379973 26061 log.cc:824] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:39:59 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:39:59.383879 26061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b: No bootstrap required, opened a new log
I20260809 05:39:59.391345 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:39:59.365945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.30.137.254:45011)
0809 05:39:59.366914 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0809 05:39:59.366930 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:39:59.366977 (+ 47us) server_negotiation.cc:407] Connection header received
0809 05:39:59.368739 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:39:59.368762 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:39:59.369162 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:39:59.369498 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:39:59.371557 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:59.372962 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:59.374984 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:39:59.375683 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:39:59.375827 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:39:59.381713 (+ 5886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:39:59.381724 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:39:59.381744 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:39:59.381793 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:39:59.386208 (+ 4415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:59.387163 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:59.387174 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:59.387182 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:59.387451 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:39:59.387829 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:39:59.387834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:39:59.387838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:39:59.388617 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:39:59.388933 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 05:39:59.389277 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 05:39:59.389798 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":435,"threads_started":1}
I20260809 05:39:59.394264 26061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45011 } }
I20260809 05:39:59.394621 26061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:39:59.394766 26061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c39e75c45c4f9ca20d2ddaa240047b, State: Initialized, Role: FOLLOWER
I20260809 05:39:59.395263 26061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45011 } }
I20260809 05:39:59.395476 26061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:39:59.395573 26061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:39:59.395707 26061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:39:59.397919 26061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45011 } }
I20260809 05:39:59.398383 26061 leader_election.cc:302] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c39e75c45c4f9ca20d2ddaa240047b; no voters:
I20260809 05:39:59.399015 26061 leader_election.cc:288] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:39:59.399170 26069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:39:59.399969 26069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [term 1 LEADER]: Becoming Leader. State: Replica: 40c39e75c45c4f9ca20d2ddaa240047b, State: Running, Role: LEADER
I20260809 05:39:59.400518 26069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45011 } }
I20260809 05:39:59.401269 26061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:39:59.404284 26071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c39e75c45c4f9ca20d2ddaa240047b. Latest consensus state: current_term: 1 leader_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45011 } } }
I20260809 05:39:59.404775 26071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:59.405177 26070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c39e75c45c4f9ca20d2ddaa240047b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45011 } } }
I20260809 05:39:59.405616 26070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b [sys.catalog]: This master's current role is: LEADER
I20260809 05:39:59.407095 26081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:39:59.413012 26081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:39:59.423477 26081 catalog_manager.cc:1347] Generated new cluster ID: f3589188c2ee4286b9f42a352224a6e2
I20260809 05:39:59.423658 26081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:39:59.440588 26081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:39:59.441988 26081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:39:59.449251 26081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b: Generated new TSK 0
I20260809 05:39:59.450261 26081 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:39:59.554122 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45011
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:39:59.920385 26095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:39:59.920714 26095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:39:59.920815 26095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:39:59.928349 26095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:39:59.928491 26095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:39:59.928610 26095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:39:59.938871 26095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:39:59.940622 26095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:39:59.941990 26095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:39:59.956576 26105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:59.956609 26107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:39:59.956727 26110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:39:59.960644 26095 server_base.cc:1034] running on GCE node
I20260809 05:39:59.961442 26095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:39:59.962754 26095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:39:59.964052 26095 hybrid_clock.cc:648] HybridClock initialized: now 1786253999963973 us; error 79 us; skew 500 ppm
I20260809 05:39:59.964453 26095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:39:59 dist-test-slave-3kk6 krb5kdc[25952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:39:59.968178 26095 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:39:59.969866 26095 webserver.cc:460] Webserver started at http://127.30.137.193:40197/ using document root <none> and password file <none>
I20260809 05:39:59.970633 26095 fs_manager.cc:359] Metadata directory not provided
I20260809 05:39:59.970836 26095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:39:59.971140 26095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:39:59.974361 26095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "6e6c66eaf60140ddaeb9540a546dcbf6"
format_stamp: "Formatted at 2026-08-09 05:39:59 on dist-test-slave-3kk6"
server_key: "335cbbefbda5cc16b7ca133afe8f1144"
server_key_iv: "a3b481a6b7e54535836f8faee59daca4"
server_key_version: "encryptionkey@0"
I20260809 05:39:59.975274 26095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "6e6c66eaf60140ddaeb9540a546dcbf6"
format_stamp: "Formatted at 2026-08-09 05:39:59 on dist-test-slave-3kk6"
server_key: "335cbbefbda5cc16b7ca133afe8f1144"
server_key_iv: "a3b481a6b7e54535836f8faee59daca4"
server_key_version: "encryptionkey@0"
I20260809 05:39:59.981074 26095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:39:59.985162 26121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:39:59.986824 26095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:39:59.986994 26095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "6e6c66eaf60140ddaeb9540a546dcbf6"
format_stamp: "Formatted at 2026-08-09 05:39:59 on dist-test-slave-3kk6"
server_key: "335cbbefbda5cc16b7ca133afe8f1144"
server_key_iv: "a3b481a6b7e54535836f8faee59daca4"
server_key_version: "encryptionkey@0"
I20260809 05:39:59.987200 26095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:00.008154 26095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:00.012676 26095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:00.012941 26095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:00.013979 26095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:00.015462 26095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:00.015540 26095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:00.015630 26095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:00.015676 26095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:00.051419 26095 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44387
I20260809 05:40:00.051461 26276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44387 every 8 connection(s)
I20260809 05:40:00.052958 26095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:40:00.059957 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 26095
I20260809 05:40:00.060163 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:40:00.060613 31271 external_mini_cluster.cc:1428] Setting key 197691c5978fe63c9de03910d4a53b6e
Aug 09 05:40:00 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253999, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:00.076260 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:00.055712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:45109 (local address 127.30.137.254:45011)
0809 05:40:00.056220 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:00.056227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:00.057335 (+ 1108us) server_negotiation.cc:407] Connection header received
0809 05:40:00.059321 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:00.059331 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:00.059403 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:00.059541 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:00.061357 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.062187 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.063977 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.064763 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.064942 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:00.068661 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:00.068670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:00.068678 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:00.068724 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:00.072520 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:00.073376 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:00.073385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:00.073390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:00.073560 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:00.074019 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:00.074026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:00.074032 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:00.074457 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:00.074620 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:00.075466 (+ 846us) server_negotiation.cc:299] Negotiation successful
0809 05:40:00.075759 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":249,"threads_started":1}
I20260809 05:40:00.076931 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:00.056359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45011 (local address 127.30.137.193:45109)
0809 05:40:00.057136 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:00.057185 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:00.059040 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:00.059773 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:00.059783 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:00.060476 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:00.061153 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:00.061173 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.062386 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.062393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:00.063803 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:00.063817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.065040 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.065048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:00.065293 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:00.066056 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:00.066086 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:00.068422 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:00.072705 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:00.072712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:00.072727 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:00.073212 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:00.073681 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:00.073684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:00.073686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:00.073889 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:00.074587 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:00.074593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:00.075052 (+ 459us) client_negotiation.cc:769] Sending connection context
0809 05:40:00.075303 (+ 251us) client_negotiation.cc:240] Negotiation successful
0809 05:40:00.075729 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":304,"threads_started":1}
I20260809 05:40:00.078888 26277 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45011
I20260809 05:40:00.079257 26277 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:00.080056 26277 heartbeater.cc:511] Master 127.30.137.254:45011 requested a full tablet report, sending...
I20260809 05:40:00.082613 26010 ts_manager.cc:194] Registered new tserver with Master: 6e6c66eaf60140ddaeb9540a546dcbf6 (127.30.137.193:44387)
I20260809 05:40:00.084913 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:45109
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:40:00.142453 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45011
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:00.508801 26285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:00.509126 26285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:00.509316 26285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:00.517047 26285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:00.517206 26285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:00.517395 26285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:40:00.527670 26285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:00.529214 26285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:00.530586 26285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:00.544749 26292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:00.544822 26293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:00.544821 26295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:00.545924 26285 server_base.cc:1034] running on GCE node
I20260809 05:40:00.546620 26285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:00.547726 26285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:00.548979 26285 hybrid_clock.cc:648] HybridClock initialized: now 1786254000548925 us; error 49 us; skew 500 ppm
I20260809 05:40:00.549341 26285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:00 dist-test-slave-3kk6 krb5kdc[25952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254000, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:00.552742 26285 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:40:00.554306 26285 webserver.cc:460] Webserver started at http://127.30.137.194:36111/ using document root <none> and password file <none>
I20260809 05:40:00.555130 26285 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:00.555297 26285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:00.555655 26285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:00.558961 26285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "640e194702b1454ebe4c35c70bd8b983"
format_stamp: "Formatted at 2026-08-09 05:40:00 on dist-test-slave-3kk6"
server_key: "e56482baa967d9b2e72174716802027f"
server_key_iv: "0522dc1cad3d73d1597e685d4925d3d7"
server_key_version: "encryptionkey@0"
I20260809 05:40:00.559859 26285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "640e194702b1454ebe4c35c70bd8b983"
format_stamp: "Formatted at 2026-08-09 05:40:00 on dist-test-slave-3kk6"
server_key: "e56482baa967d9b2e72174716802027f"
server_key_iv: "0522dc1cad3d73d1597e685d4925d3d7"
server_key_version: "encryptionkey@0"
I20260809 05:40:00.565549 26285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 05:40:00.569355 26305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:00.570937 26285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:00.571138 26285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "640e194702b1454ebe4c35c70bd8b983"
format_stamp: "Formatted at 2026-08-09 05:40:00 on dist-test-slave-3kk6"
server_key: "e56482baa967d9b2e72174716802027f"
server_key_iv: "0522dc1cad3d73d1597e685d4925d3d7"
server_key_version: "encryptionkey@0"
I20260809 05:40:00.571388 26285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:00.593415 26285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:00.598016 26285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:00.598352 26285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:00.599519 26285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:00.601027 26285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:00.601150 26285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:00.601373 26285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:00.601454 26285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:00.637940 26285 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:45823
I20260809 05:40:00.638002 26474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:45823 every 8 connection(s)
I20260809 05:40:00.639394 26285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:40:00.649096 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 26285
I20260809 05:40:00.649326 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:40:00.649803 31271 external_mini_cluster.cc:1428] Setting key cf4ea890834df398cd0b5e5b42282855
Aug 09 05:40:00 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254000, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:00.663069 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:00.642160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:39783 (local address 127.30.137.254:45011)
0809 05:40:00.642626 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:00.642631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:00.643800 (+ 1169us) server_negotiation.cc:407] Connection header received
0809 05:40:00.645799 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:00.645806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:00.645904 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:00.646035 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:00.647795 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.648540 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.650370 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.650942 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.651104 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:00.655411 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:00.655420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:00.655428 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:00.655478 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:00.659258 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:00.660116 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:00.660124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:00.660130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:00.660306 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:00.660886 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:00.660894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:00.660899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:00.661309 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:00.661475 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:00.662252 (+ 777us) server_negotiation.cc:299] Negotiation successful
0809 05:40:00.662524 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":220,"threads_started":1}
I20260809 05:40:00.663789 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:00.642817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45011 (local address 127.30.137.194:39783)
0809 05:40:00.643606 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:00.643647 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:00.645487 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:00.646324 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:00.646333 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:00.646998 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:00.647598 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:00.647616 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.648719 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.648724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:00.650208 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:00.650223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:00.651180 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:00.651185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:00.651397 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:00.652259 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:00.652300 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:00.655143 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:00.659448 (+ 4305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:00.659455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:00.659470 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:00.659963 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:00.660458 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:00.660462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:00.660465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:00.660722 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:00.661447 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:00.661454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:00.661805 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 05:40:00.662080 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 05:40:00.662508 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":319,"threads_started":1}
I20260809 05:40:00.665364 26475 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45011
I20260809 05:40:00.665730 26475 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:00.666682 26475 heartbeater.cc:511] Master 127.30.137.254:45011 requested a full tablet report, sending...
I20260809 05:40:00.668584 26010 ts_manager.cc:194] Registered new tserver with Master: 640e194702b1454ebe4c35c70bd8b983 (127.30.137.194:45823)
I20260809 05:40:00.669536 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:39783
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:40:00.727913 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45011
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:43727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:40:01.088418 26277 heartbeater.cc:503] Master 127.30.137.254:45011 was elected leader, sending a full tablet report...
W20260809 05:40:01.103569 26484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:01.103965 26484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:01.104151 26484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:01.112067 26484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:01.112267 26484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:01.112473 26484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:40:01.123011 26484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45011
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:01.124497 26484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:01.125907 26484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:01.141290 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:01.140985 26495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:01.141299 26502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:01.142479 26484 server_base.cc:1034] running on GCE node
I20260809 05:40:01.143239 26484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:01.144418 26484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:01.145704 26484 hybrid_clock.cc:648] HybridClock initialized: now 1786254001145625 us; error 75 us; skew 500 ppm
I20260809 05:40:01.146100 26484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:01 dist-test-slave-3kk6 krb5kdc[25952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:01.150015 26484 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:40:01.151829 26484 webserver.cc:460] Webserver started at http://127.30.137.195:34969/ using document root <none> and password file <none>
I20260809 05:40:01.152642 26484 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:01.152815 26484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:01.153160 26484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:01.156576 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "8317da1bfe4d4914ae9cba106e89eeb0"
format_stamp: "Formatted at 2026-08-09 05:40:01 on dist-test-slave-3kk6"
server_key: "5d3d4c218540508cd938cc8449b12f7d"
server_key_iv: "b10b6bdf6de6f2a1d4b56076f6d92c70"
server_key_version: "encryptionkey@0"
I20260809 05:40:01.157492 26484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "8317da1bfe4d4914ae9cba106e89eeb0"
format_stamp: "Formatted at 2026-08-09 05:40:01 on dist-test-slave-3kk6"
server_key: "5d3d4c218540508cd938cc8449b12f7d"
server_key_iv: "b10b6bdf6de6f2a1d4b56076f6d92c70"
server_key_version: "encryptionkey@0"
I20260809 05:40:01.163954 26484 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260809 05:40:01.167976 26511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:01.169672 26484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:40:01.169929 26484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "8317da1bfe4d4914ae9cba106e89eeb0"
format_stamp: "Formatted at 2026-08-09 05:40:01 on dist-test-slave-3kk6"
server_key: "5d3d4c218540508cd938cc8449b12f7d"
server_key_iv: "b10b6bdf6de6f2a1d4b56076f6d92c70"
server_key_version: "encryptionkey@0"
I20260809 05:40:01.170197 26484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:01.193820 26484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:01.198405 26484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:01.198807 26484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:01.199908 26484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:01.201535 26484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:01.201656 26484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:01.201815 26484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:01.201892 26484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:01.238138 26484 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:39233
I20260809 05:40:01.238201 26665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:39233 every 8 connection(s)
I20260809 05:40:01.239639 26484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:40:01.244205 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 26484
I20260809 05:40:01.244405 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:40:01.244865 31271 external_mini_cluster.cc:1428] Setting key 7717660baf6a7aa6f312e6ae639b0557
Aug 09 05:40:01 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:01.261492 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.242349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:58693 (local address 127.30.137.254:45011)
0809 05:40:01.242924 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.242929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.243781 (+ 852us) server_negotiation.cc:407] Connection header received
0809 05:40:01.245975 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.245981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.246105 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.246247 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:01.248325 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.249131 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.250721 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.251138 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.251237 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.254884 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:01.254889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:01.254892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:01.254929 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:01.257782 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.258790 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.258797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.258803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.258987 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.259554 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.259558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.259560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.259950 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:01.260079 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.261018 (+ 939us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.261199 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":264,"threads_started":1}
I20260809 05:40:01.262494 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.242825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45011 (local address 127.30.137.195:58693)
0809 05:40:01.243561 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:01.243609 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:01.245695 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:01.246564 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:01.246576 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:01.247372 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:01.248122 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.248158 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.249269 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.249280 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.250525 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.250537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.251343 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.251349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.251549 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.252243 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:01.252266 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:01.254576 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:01.257935 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:01.257943 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:01.257957 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:01.258605 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:01.259112 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:01.259119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:01.259126 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:01.259419 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:01.260090 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:01.260099 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:01.260554 (+ 455us) client_negotiation.cc:769] Sending connection context
0809 05:40:01.260892 (+ 338us) client_negotiation.cc:240] Negotiation successful
0809 05:40:01.261329 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":297,"threads_started":1}
I20260809 05:40:01.263885 26666 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45011
I20260809 05:40:01.264225 26666 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:01.264977 26666 heartbeater.cc:511] Master 127.30.137.254:45011 requested a full tablet report, sending...
I20260809 05:40:01.266384 26010 ts_manager.cc:194] Registered new tserver with Master: 8317da1bfe4d4914ae9cba106e89eeb0 (127.30.137.195:39233)
I20260809 05:40:01.267236 26010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:58693
I20260809 05:40:01.273634 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:40:01.293582 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.278401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.30.137.254:45011)
0809 05:40:01.278591 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.278595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.279085 (+ 490us) server_negotiation.cc:407] Connection header received
0809 05:40:01.279252 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.279256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.279343 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.279443 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:01.280982 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.281768 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.283302 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.283769 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.283892 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.287050 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:01.287056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:01.287060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:01.287108 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:01.290404 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.291416 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.291423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.291429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.291602 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.292172 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.292176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.292178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.292557 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:01.292788 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.293159 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.293343 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:40:01.297995 26010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:40:01.305735 26010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:40:01.342837 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.319595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.30.137.195:39233)
0809 05:40:01.331343 (+ 11748us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.331359 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.332750 (+ 1391us) server_negotiation.cc:407] Connection header received
0809 05:40:01.332935 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.332944 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.333160 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.333325 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:01.334626 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.336511 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.339952 (+ 3441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.341392 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.341625 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.341830 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.342174 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.342399 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11512,"thread_start_us":11325,"threads_started":1}
I20260809 05:40:01.343269 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.319334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.30.137.193:44387)
0809 05:40:01.331819 (+ 12485us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.331833 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.332452 (+ 619us) server_negotiation.cc:407] Connection header received
0809 05:40:01.332706 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.332716 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.333045 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.333277 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:01.334750 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.336483 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.340274 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.341814 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.342041 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.342249 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.342695 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.342924 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12211,"thread_start_us":12500,"threads_started":1}
I20260809 05:40:01.343852 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.318986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44387 (local address 127.0.0.1:36604)
0809 05:40:01.332071 (+ 13085us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:01.332128 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:01.332367 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:01.333628 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:01.333636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:01.333694 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:01.334487 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.334502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.336723 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.336728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.340044 (+ 3316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.340057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.342260 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.342270 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.342447 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.342475 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 05:40:01.342597 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 05:40:01.343114 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12794,"mutex_wait_us":1169,"spinlock_wait_cycles":8832,"thread_start_us":13270,"threads_started":1}
I20260809 05:40:01.343878 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.319202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:39233 (local address 127.0.0.1:43406)
0809 05:40:01.332586 (+ 13384us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:01.332609 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:01.332761 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:01.333627 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:01.333636 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:01.333695 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:01.334340 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.334362 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.336723 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.336728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.339726 (+ 2998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.339746 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.341627 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.341635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.341840 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.341906 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 05:40:01.343095 (+ 1189us) client_negotiation.cc:240] Negotiation successful
0809 05:40:01.343260 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13259,"mutex_wait_us":36,"spinlock_wait_cycles":9984,"thread_start_us":12405,"threads_started":1}
I20260809 05:40:01.348218 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.335151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:45823 (local address 127.0.0.1:54208)
0809 05:40:01.337849 (+ 2698us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:01.337873 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:01.338070 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:01.338865 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:01.338869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:01.338898 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:01.339470 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.339485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.342077 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.342083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.345001 (+ 2918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.345017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.347368 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.347386 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.347567 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.347604 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:40:01.347715 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 05:40:01.347866 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2564,"thread_start_us":2331,"threads_started":1}
I20260809 05:40:01.348328 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.333722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.30.137.194:45823)
0809 05:40:01.337232 (+ 3510us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.337242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.338007 (+ 765us) server_negotiation.cc:407] Connection header received
0809 05:40:01.338268 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.338275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.338581 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.340009 (+ 1428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:01.340293 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.341921 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.345194 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.346858 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.347031 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.347214 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.347810 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.348002 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3292,"thread_start_us":3220,"threads_started":1}
I20260809 05:40:01.348496 26573 tablet_service.cc:1467] Processing CreateTablet for tablet 70fad38fbc4545c890a3ce666b991c28 (DEFAULT_TABLE table=test-table [id=1f2ace67e10346d5b477542f231a996f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:01.348719 26186 tablet_service.cc:1467] Processing CreateTablet for tablet 70fad38fbc4545c890a3ce666b991c28 (DEFAULT_TABLE table=test-table [id=1f2ace67e10346d5b477542f231a996f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:01.350010 26186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70fad38fbc4545c890a3ce666b991c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:01.350023 26573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70fad38fbc4545c890a3ce666b991c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:01.357451 26380 tablet_service.cc:1467] Processing CreateTablet for tablet 70fad38fbc4545c890a3ce666b991c28 (DEFAULT_TABLE table=test-table [id=1f2ace67e10346d5b477542f231a996f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:01.359198 26380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70fad38fbc4545c890a3ce666b991c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:01.370831 26694 tablet_bootstrap.cc:492] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Bootstrap starting.
I20260809 05:40:01.370841 26693 tablet_bootstrap.cc:492] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Bootstrap starting.
I20260809 05:40:01.374861 26693 tablet_bootstrap.cc:654] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:01.376988 26694 tablet_bootstrap.cc:654] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:01.378109 26693 log.cc:824] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:01.378237 26694 log.cc:824] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:01.381224 26696 tablet_bootstrap.cc:492] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Bootstrap starting.
I20260809 05:40:01.387733 26696 tablet_bootstrap.cc:654] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:01.387796 26693 tablet_bootstrap.cc:492] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: No bootstrap required, opened a new log
I20260809 05:40:01.387799 26694 tablet_bootstrap.cc:492] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: No bootstrap required, opened a new log
I20260809 05:40:01.388103 26693 ts_tablet_manager.cc:1397] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.010s
I20260809 05:40:01.388121 26694 ts_tablet_manager.cc:1397] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.000s
I20260809 05:40:01.389389 26696 log.cc:824] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:01.393250 26696 tablet_bootstrap.cc:492] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: No bootstrap required, opened a new log
I20260809 05:40:01.393635 26696 ts_tablet_manager.cc:1397] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260809 05:40:01.396273 26694 raft_consensus.cc:348] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.396682 26694 raft_consensus.cc:374] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:01.396790 26694 raft_consensus.cc:729] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8317da1bfe4d4914ae9cba106e89eeb0, State: Initialized, Role: FOLLOWER
I20260809 05:40:01.397462 26694 consensus_queue.cc:260] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.398727 26666 heartbeater.cc:503] Master 127.30.137.254:45011 was elected leader, sending a full tablet report...
I20260809 05:40:01.399330 26694 ts_tablet_manager.cc:1428] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 05:40:01.400007 26693 raft_consensus.cc:348] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.400424 26693 raft_consensus.cc:374] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:01.400550 26693 raft_consensus.cc:729] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6c66eaf60140ddaeb9540a546dcbf6, State: Initialized, Role: FOLLOWER
I20260809 05:40:01.401196 26693 consensus_queue.cc:260] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.404582 26696 raft_consensus.cc:348] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.404974 26693 ts_tablet_manager.cc:1428] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Time spent starting tablet: real 0.017s user 0.007s sys 0.009s
I20260809 05:40:01.405073 26696 raft_consensus.cc:374] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:01.405260 26696 raft_consensus.cc:729] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640e194702b1454ebe4c35c70bd8b983, State: Initialized, Role: FOLLOWER
I20260809 05:40:01.406068 26696 consensus_queue.cc:260] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.407482 26475 heartbeater.cc:503] Master 127.30.137.254:45011 was elected leader, sending a full tablet report...
I20260809 05:40:01.408219 26696 ts_tablet_manager.cc:1428] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260809 05:40:01.491513 26667 tablet.cc:2369] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:01.517017 26702 raft_consensus.cc:482] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:40:01.517378 26702 raft_consensus.cc:504] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.519405 26702 leader_election.cc:288] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8317da1bfe4d4914ae9cba106e89eeb0 (127.30.137.195:39233), 640e194702b1454ebe4c35c70bd8b983 (127.30.137.194:45823)
I20260809 05:40:01.526543 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.519912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:37759 (local address 127.30.137.194:45823)
0809 05:40:01.520187 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.520195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.520886 (+ 691us) server_negotiation.cc:407] Connection header received
0809 05:40:01.521123 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.521127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.521189 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.521299 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:01.521900 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.522672 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.524687 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.525489 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.525680 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.525778 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.526112 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.526245 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 05:40:01.526573 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.519803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:36031 (local address 127.30.137.195:39233)
0809 05:40:01.520053 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.520060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.520943 (+ 883us) server_negotiation.cc:407] Connection header received
0809 05:40:01.521151 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.521155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.521218 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.521325 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:01.522039 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.522900 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.524747 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.525528 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.525727 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.525811 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.526167 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.526299 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 05:40:01.526779 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.519864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:39233 (local address 127.30.137.193:36031)
0809 05:40:01.520787 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:01.520802 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:01.520986 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:01.521339 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:01.521344 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:01.521376 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:01.521870 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.521880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.523052 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.523057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.524566 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.524579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.525781 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.525788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.526000 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.526024 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:40:01.526138 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 05:40:01.526271 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":67,"thread_start_us":560,"threads_started":1}
I20260809 05:40:01.527184 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.519685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:45823 (local address 127.30.137.193:37759)
0809 05:40:01.520738 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:01.520759 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:01.520953 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:01.521321 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:01.521325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:01.521351 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:01.521736 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.521749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.522859 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.522866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.524501 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:01.524517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.525736 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.525742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:01.525940 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.525967 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 05:40:01.526080 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 05:40:01.526233 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":41,"thread_start_us":640,"threads_started":1}
I20260809 05:40:01.527860 26601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fad38fbc4545c890a3ce666b991c28" candidate_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" is_pre_election: true
I20260809 05:40:01.528412 26601 raft_consensus.cc:2454] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e6c66eaf60140ddaeb9540a546dcbf6 in term 0.
I20260809 05:40:01.528471 26414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fad38fbc4545c890a3ce666b991c28" candidate_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640e194702b1454ebe4c35c70bd8b983" is_pre_election: true
I20260809 05:40:01.529055 26414 raft_consensus.cc:2454] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e6c66eaf60140ddaeb9540a546dcbf6 in term 0.
I20260809 05:40:01.529162 26125 leader_election.cc:302] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e6c66eaf60140ddaeb9540a546dcbf6, 8317da1bfe4d4914ae9cba106e89eeb0; no voters:
I20260809 05:40:01.529628 26702 raft_consensus.cc:2781] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:40:01.529785 26702 raft_consensus.cc:482] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:40:01.529928 26702 raft_consensus.cc:3037] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:01.532401 26702 raft_consensus.cc:504] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.533126 26702 leader_election.cc:288] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [CANDIDATE]: Term 1 election: Requested vote from peers 8317da1bfe4d4914ae9cba106e89eeb0 (127.30.137.195:39233), 640e194702b1454ebe4c35c70bd8b983 (127.30.137.194:45823)
I20260809 05:40:01.533612 26601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fad38fbc4545c890a3ce666b991c28" candidate_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8317da1bfe4d4914ae9cba106e89eeb0"
I20260809 05:40:01.533706 26414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fad38fbc4545c890a3ce666b991c28" candidate_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "640e194702b1454ebe4c35c70bd8b983"
I20260809 05:40:01.533859 26601 raft_consensus.cc:3037] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:01.533934 26414 raft_consensus.cc:3037] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:01.537395 26601 raft_consensus.cc:2454] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e6c66eaf60140ddaeb9540a546dcbf6 in term 1.
I20260809 05:40:01.537389 26414 raft_consensus.cc:2454] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e6c66eaf60140ddaeb9540a546dcbf6 in term 1.
I20260809 05:40:01.537950 26124 leader_election.cc:302] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 640e194702b1454ebe4c35c70bd8b983, 6e6c66eaf60140ddaeb9540a546dcbf6; no voters:
I20260809 05:40:01.538316 26702 raft_consensus.cc:2781] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:01.538900 26702 raft_consensus.cc:686] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 1 LEADER]: Becoming Leader. State: Replica: 6e6c66eaf60140ddaeb9540a546dcbf6, State: Running, Role: LEADER
I20260809 05:40:01.539505 26702 consensus_queue.cc:237] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } }
I20260809 05:40:01.544814 26008 catalog_manager.cc:5495] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e6c66eaf60140ddaeb9540a546dcbf6 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e6c66eaf60140ddaeb9540a546dcbf6" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 } health_report { overall_health: UNKNOWN } } }
W20260809 05:40:01.555827 26279 tablet.cc:2369] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:01.578961 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.573085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.30.137.193:44387)
0809 05:40:01.573278 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.573283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.573301 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:40:01.573393 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.573397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.573500 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.573631 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:40:01.574054 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.574923 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.576580 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.576996 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.577146 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.577751 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:40:01.578275 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:40:01.578479 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.578616 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.578702 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:40:01.587826 26601 raft_consensus.cc:1261] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 1 FOLLOWER]: Refusing update from remote peer 6e6c66eaf60140ddaeb9540a546dcbf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:01.587877 26414 raft_consensus.cc:1261] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 1 FOLLOWER]: Refusing update from remote peer 6e6c66eaf60140ddaeb9540a546dcbf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:01.588886 26708 consensus_queue.cc:1035] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "640e194702b1454ebe4c35c70bd8b983" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:01.589291 26702 consensus_queue.cc:1035] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8317da1bfe4d4914ae9cba106e89eeb0" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:01.601953 26713 mvcc.cc:205] Tried to move back new op lower bound from 7316496390492192768 to 7316496390310330368. Current Snapshot: MvccSnapshot[applied={T|T < 7316496390492192768}]
I20260809 05:40:01.625069 26008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:01.625416 26008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:01.629761 26008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40c39e75c45c4f9ca20d2ddaa240047b: Sending DeleteTablet for 3 replicas of tablet 70fad38fbc4545c890a3ce666b991c28
I20260809 05:40:01.631492 26573 tablet_service.cc:1514] Processing DeleteTablet for tablet 70fad38fbc4545c890a3ce666b991c28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:01 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:43406
I20260809 05:40:01.631615 26380 tablet_service.cc:1514] Processing DeleteTablet for tablet 70fad38fbc4545c890a3ce666b991c28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:01 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:54208
I20260809 05:40:01.632707 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 26095
I20260809 05:40:01.632812 26186 tablet_service.cc:1514] Processing DeleteTablet for tablet 70fad38fbc4545c890a3ce666b991c28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:01 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:36604
I20260809 05:40:01.646994 26731 tablet_replica.cc:330] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: stopping tablet replica
I20260809 05:40:01.647586 26731 raft_consensus.cc:2229] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:40:01.648344 26731 raft_consensus.cc:2258] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:01.653774 26731 ts_tablet_manager.cc:1905] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:01.663792 26731 ts_tablet_manager.cc:1918] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:01.664053 26731 log.cc:1196] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/70fad38fbc4545c890a3ce666b991c28
I20260809 05:40:01.664808 26731 ts_tablet_manager.cc:1939] T 70fad38fbc4545c890a3ce666b991c28 P 6e6c66eaf60140ddaeb9540a546dcbf6: Deleting consensus metadata
I20260809 05:40:01.665515 26729 tablet_replica.cc:330] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: stopping tablet replica
I20260809 05:40:01.667078 25989 catalog_manager.cc:4841] TS 6e6c66eaf60140ddaeb9540a546dcbf6 (127.30.137.193:44387): tablet 70fad38fbc4545c890a3ce666b991c28 (table test-table [id=1f2ace67e10346d5b477542f231a996f]) successfully deleted
I20260809 05:40:01.668035 26729 raft_consensus.cc:2229] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:01.671497 26729 raft_consensus.cc:2258] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 05:40:01 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:40:01.677696 26730 tablet_replica.cc:330] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: stopping tablet replica
I20260809 05:40:01.678221 26730 raft_consensus.cc:2229] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:01.678841 26730 raft_consensus.cc:2258] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:01.680428 26729 ts_tablet_manager.cc:1905] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:01.685478 26730 ts_tablet_manager.cc:1905] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:01.688446 26729 ts_tablet_manager.cc:1918] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:01.688731 26729 log.cc:1196] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/70fad38fbc4545c890a3ce666b991c28
I20260809 05:40:01.689409 26729 ts_tablet_manager.cc:1939] T 70fad38fbc4545c890a3ce666b991c28 P 8317da1bfe4d4914ae9cba106e89eeb0: Deleting consensus metadata
I20260809 05:40:01.689661 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.634549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.30.137.193:44387)
0809 05:40:01.634819 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.634828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.659849 (+ 25021us) server_negotiation.cc:407] Connection header received
0809 05:40:01.661002 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.661009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.661112 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.661209 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:01.665896 (+ 4687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.667216 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.669459 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.670160 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.670260 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.679377 (+ 9117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:01.679400 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:01.679424 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:01.679477 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:01.684043 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.686461 (+ 2418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.686471 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.686475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.686747 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.687330 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.687349 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.687353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.687897 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:01.688215 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.688910 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.689230 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:40:01.691756 26268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:01.692195 25990 catalog_manager.cc:4841] TS 8317da1bfe4d4914ae9cba106e89eeb0 (127.30.137.195:39233): tablet 70fad38fbc4545c890a3ce666b991c28 (table test-table [id=1f2ace67e10346d5b477542f231a996f]) successfully deleted
I20260809 05:40:01.693593 26730 ts_tablet_manager.cc:1918] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:01.693830 26730 log.cc:1196] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/70fad38fbc4545c890a3ce666b991c28
I20260809 05:40:01.694569 26730 ts_tablet_manager.cc:1939] T 70fad38fbc4545c890a3ce666b991c28 P 640e194702b1454ebe4c35c70bd8b983: Deleting consensus metadata
I20260809 05:40:01.697206 25989 catalog_manager.cc:4841] TS 640e194702b1454ebe4c35c70bd8b983 (127.30.137.194:45823): tablet 70fad38fbc4545c890a3ce666b991c28 (table test-table [id=1f2ace67e10346d5b477542f231a996f]) successfully deleted
I20260809 05:40:01.798030 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 26095
I20260809 05:40:01.826470 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 26285
Aug 09 05:40:01 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:40:01.843708 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.827496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.30.137.194:45823)
0809 05:40:01.827745 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.827751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.827775 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:40:01.828013 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.828017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.828086 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.828202 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:01.829606 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.830434 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.832059 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.832483 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.832635 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.837335 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:01.837354 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:01.837371 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:01.837416 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:01.840708 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.841503 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.841512 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.841514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.841658 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.842135 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.842138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.842141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.842560 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:01.842888 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.843149 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.843344 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 05:40:01.844573 26463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:01.938825 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 26285
I20260809 05:40:01.965811 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 26484
Aug 09 05:40:01 dist-test-slave-3kk6 krb5kdc[25952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786253998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:40:01.982801 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:01.966723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.30.137.195:39233)
0809 05:40:01.966956 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:01.966964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:01.966985 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:40:01.967203 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:01.967208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:01.967276 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:01.967410 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:01.968830 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.969581 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.971049 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:01.971468 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:01.971627 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:01.976669 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:01.976687 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:01.976704 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:01.976745 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:01.979616 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.980440 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.980449 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.980451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.980606 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:01.981037 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:01.981047 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:01.981049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:01.981434 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:01.981759 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:01.982119 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 05:40:01.982372 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:40:01.983526 26660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:02.077895 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 26484
I20260809 05:40:02.111603 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 25970
I20260809 05:40:02.112862 26054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:02.249714 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 25970
2026-08-09T05:40:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5080 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T05:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:02Z Disabled control of system clock
I20260809 05:40:02.361852 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42879
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:42879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:40:02.742097 26745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:02.742457 26745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:02.742545 26745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:02.750281 26745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:02.750435 26745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:02.750475 26745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:02.750541 26745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:02.750587 26745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:02.760473 26745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:42879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:42879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:02.761861 26745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:02.763780 26745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:02.776782 26751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:02.776844 26752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:02.776876 26754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:02.777787 26745 server_base.cc:1034] running on GCE node
I20260809 05:40:02.778765 26745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:02.780385 26745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:02.781685 26745 hybrid_clock.cc:648] HybridClock initialized: now 1786254002781637 us; error 46 us; skew 500 ppm
I20260809 05:40:02.782131 26745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:02.784806 26745 webserver.cc:460] Webserver started at http://127.30.137.254:46463/ using document root <none> and password file <none>
I20260809 05:40:02.785588 26745 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:02.785751 26745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:02.786110 26745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:02.789487 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "f800ce1c066249a584349233b41714f6"
format_stamp: "Formatted at 2026-08-09 05:40:02 on dist-test-slave-3kk6"
server_key: "d6c490a1353ff08bfd5a1c50617d4368"
server_key_iv: "92fc247819e951c73baf7895ddcc7ecb"
server_key_version: "encryptionkey@0"
I20260809 05:40:02.790459 26745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "f800ce1c066249a584349233b41714f6"
format_stamp: "Formatted at 2026-08-09 05:40:02 on dist-test-slave-3kk6"
server_key: "d6c490a1353ff08bfd5a1c50617d4368"
server_key_iv: "92fc247819e951c73baf7895ddcc7ecb"
server_key_version: "encryptionkey@0"
I20260809 05:40:02.796440 26745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:40:02.800623 26764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:02.802374 26745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:40:02.802631 26745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "f800ce1c066249a584349233b41714f6"
format_stamp: "Formatted at 2026-08-09 05:40:02 on dist-test-slave-3kk6"
server_key: "d6c490a1353ff08bfd5a1c50617d4368"
server_key_iv: "92fc247819e951c73baf7895ddcc7ecb"
server_key_version: "encryptionkey@0"
I20260809 05:40:02.803021 26745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:02.827582 26745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:02.828617 26745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:02.829003 26745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:02.850898 26745 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:42879
I20260809 05:40:02.850870 26857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:42879 every 8 connection(s)
I20260809 05:40:02.852502 26745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:02.856860 26859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:02.858688 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 26745
I20260809 05:40:02.858899 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:02.859531 31271 external_mini_cluster.cc:1428] Setting key fceeba8b1f15daa1d770367a4b576942
I20260809 05:40:02.870453 26859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6: Bootstrap starting.
I20260809 05:40:02.874502 26859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:02.875680 26859 log.cc:824] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:02.875708 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:02.861791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.30.137.254:42879)
0809 05:40:02.863051 (+ 1260us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:02.863066 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:02.863120 (+ 54us) server_negotiation.cc:407] Connection header received
0809 05:40:02.865056 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:02.865080 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:02.865582 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:02.865971 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:02.866809 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:02.868349 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:02.870398 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:02.871339 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:02.871493 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:02.872936 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:02.872944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:02.872959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:02.873575 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:02.873687 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:02.873716 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:02.873807 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:02.874102 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 05:40:02.874496 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":486,"threads_started":1}
I20260809 05:40:02.879078 26859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6: No bootstrap required, opened a new log
I20260809 05:40:02.886756 26859 raft_consensus.cc:348] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f800ce1c066249a584349233b41714f6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42879 } }
I20260809 05:40:02.887240 26859 raft_consensus.cc:374] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:02.887393 26859 raft_consensus.cc:729] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f800ce1c066249a584349233b41714f6, State: Initialized, Role: FOLLOWER
I20260809 05:40:02.888072 26859 consensus_queue.cc:260] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f800ce1c066249a584349233b41714f6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42879 } }
I20260809 05:40:02.888360 26859 raft_consensus.cc:388] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:02.888527 26859 raft_consensus.cc:482] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:02.888720 26859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:02.891186 26859 raft_consensus.cc:504] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f800ce1c066249a584349233b41714f6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42879 } }
I20260809 05:40:02.891815 26859 leader_election.cc:302] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f800ce1c066249a584349233b41714f6; no voters:
I20260809 05:40:02.892601 26859 leader_election.cc:288] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:02.892746 26867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:02.893488 26867 raft_consensus.cc:686] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [term 1 LEADER]: Becoming Leader. State: Replica: f800ce1c066249a584349233b41714f6, State: Running, Role: LEADER
I20260809 05:40:02.894109 26867 consensus_queue.cc:237] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f800ce1c066249a584349233b41714f6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42879 } }
I20260809 05:40:02.895220 26859 sys_catalog.cc:564] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:02.897457 26868 sys_catalog.cc:455] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f800ce1c066249a584349233b41714f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f800ce1c066249a584349233b41714f6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42879 } } }
I20260809 05:40:02.897907 26868 sys_catalog.cc:458] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:02.898298 26869 sys_catalog.cc:455] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f800ce1c066249a584349233b41714f6. Latest consensus state: current_term: 1 leader_uuid: "f800ce1c066249a584349233b41714f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f800ce1c066249a584349233b41714f6" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 42879 } } }
I20260809 05:40:02.898705 26869 sys_catalog.cc:458] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:02.899899 26875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:02.907598 26875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:02.917855 26875 catalog_manager.cc:1347] Generated new cluster ID: 3c1ec6c21d334aa1b589ae63862c0e63
I20260809 05:40:02.918025 26875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:02.935421 26875 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:02.937544 26875 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:02.959582 26875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6: Generated new TSK 0
I20260809 05:40:02.960639 26875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:02.976946 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42879
--builtin_ntp_servers=127.30.137.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:40:03.378247 26897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:03.378589 26897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:03.378800 26897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:03.386791 26897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:03.386929 26897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:03.387151 26897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:40:03.397364 26897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:03.399019 26897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:03.401055 26897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:03.416085 26903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:03.416013 26910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:03.416124 26906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:03.416987 26897 server_base.cc:1034] running on GCE node
I20260809 05:40:03.417591 26897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:03.418560 26897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:03.419781 26897 hybrid_clock.cc:648] HybridClock initialized: now 1786254003419728 us; error 47 us; skew 500 ppm
I20260809 05:40:03.420153 26897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:03.422819 26897 webserver.cc:460] Webserver started at http://127.30.137.193:33393/ using document root <none> and password file <none>
I20260809 05:40:03.423588 26897 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:03.423723 26897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:03.424073 26897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:03.427229 26897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "6329110dde1f4eeaa59ec4feb43eab2c"
format_stamp: "Formatted at 2026-08-09 05:40:03 on dist-test-slave-3kk6"
server_key: "ee4be9a0171f7f8d8470e54daf87df7c"
server_key_iv: "8e7aebb6344b282ec666e28922456101"
server_key_version: "encryptionkey@0"
I20260809 05:40:03.428104 26897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "6329110dde1f4eeaa59ec4feb43eab2c"
format_stamp: "Formatted at 2026-08-09 05:40:03 on dist-test-slave-3kk6"
server_key: "ee4be9a0171f7f8d8470e54daf87df7c"
server_key_iv: "8e7aebb6344b282ec666e28922456101"
server_key_version: "encryptionkey@0"
I20260809 05:40:03.433715 26897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:40:03.437718 26922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:03.439607 26897 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:40:03.439869 26897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "6329110dde1f4eeaa59ec4feb43eab2c"
format_stamp: "Formatted at 2026-08-09 05:40:03 on dist-test-slave-3kk6"
server_key: "ee4be9a0171f7f8d8470e54daf87df7c"
server_key_iv: "8e7aebb6344b282ec666e28922456101"
server_key_version: "encryptionkey@0"
I20260809 05:40:03.440200 26897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:03.456269 26897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:03.457218 26897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:03.457573 26897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:03.458745 26897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:03.460528 26897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:03.460665 26897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:03.460806 26897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:03.460901 26897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:03.498251 26897 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:39763
I20260809 05:40:03.498304 27091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:39763 every 8 connection(s)
I20260809 05:40:03.499788 26897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:40:03.501958 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 26897
I20260809 05:40:03.502166 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:40:03.502871 31271 external_mini_cluster.cc:1428] Setting key c461c38a3d3555a7ae5acf6785adf556
I20260809 05:40:03.506213 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42879
--builtin_ntp_servers=127.30.137.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:40:03.515218 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:03.503128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:35203 (local address 127.30.137.254:42879)
0809 05:40:03.503725 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:03.503731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:03.505202 (+ 1471us) server_negotiation.cc:407] Connection header received
0809 05:40:03.507383 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:03.507392 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:03.507493 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:03.507624 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:03.508881 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:03.510055 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:03.511811 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:03.512508 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:03.512657 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:03.514134 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:03.514144 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:03.514151 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:03.514178 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:03.514234 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:03.514253 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:03.514351 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:03.514585 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 05:40:03.514751 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":294,"threads_started":1}
I20260809 05:40:03.516126 27095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:03.503822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42879 (local address 127.30.137.193:35203)
0809 05:40:03.505012 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:03.505071 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:03.507090 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:03.507903 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:03.507911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:03.508007 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:03.508677 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:03.508702 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:03.510236 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:03.510240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:03.511632 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:03.511644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:03.512764 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:03.512770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:03.512976 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:03.513790 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:40:03.513793 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:03.513835 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:40:03.513918 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:03.514364 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:03.514373 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:03.514379 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:40:03.514537 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 05:40:03.514949 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":310,"threads_started":1}
I20260809 05:40:03.517899 27092 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42879
I20260809 05:40:03.518249 27092 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:03.519272 27092 heartbeater.cc:511] Master 127.30.137.254:42879 requested a full tablet report, sending...
I20260809 05:40:03.522091 26791 ts_manager.cc:194] Registered new tserver with Master: 6329110dde1f4eeaa59ec4feb43eab2c (127.30.137.193:39763)
I20260809 05:40:03.523850 26791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.193:35203
W20260809 05:40:03.881027 27096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:03.881443 27096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:03.881640 27096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:03.889496 27096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:03.889699 27096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:03.889916 27096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:40:03.899948 27096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:03.901607 27096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:03.903813 27096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:03.919061 27101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:03.919073 27102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:03.919288 27111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:03.920006 27096 server_base.cc:1034] running on GCE node
I20260809 05:40:03.920751 27096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:03.921893 27096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:03.923161 27096 hybrid_clock.cc:648] HybridClock initialized: now 1786254003923106 us; error 48 us; skew 500 ppm
I20260809 05:40:03.923581 27096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:03.926465 27096 webserver.cc:460] Webserver started at http://127.30.137.194:34293/ using document root <none> and password file <none>
I20260809 05:40:03.927284 27096 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:03.927394 27096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:03.927678 27096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:03.931001 27096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "18391f1408bd410a92f2c6b51b496234"
format_stamp: "Formatted at 2026-08-09 05:40:03 on dist-test-slave-3kk6"
server_key: "e36c2ac1da08651c2f32dbb928e9ed35"
server_key_iv: "83f760224f728af2eae03ed87654c7a5"
server_key_version: "encryptionkey@0"
I20260809 05:40:03.931939 27096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "18391f1408bd410a92f2c6b51b496234"
format_stamp: "Formatted at 2026-08-09 05:40:03 on dist-test-slave-3kk6"
server_key: "e36c2ac1da08651c2f32dbb928e9ed35"
server_key_iv: "83f760224f728af2eae03ed87654c7a5"
server_key_version: "encryptionkey@0"
I20260809 05:40:03.938186 27096 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 05:40:03.942340 27121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:03.944097 27096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:03.944324 27096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "18391f1408bd410a92f2c6b51b496234"
format_stamp: "Formatted at 2026-08-09 05:40:03 on dist-test-slave-3kk6"
server_key: "e36c2ac1da08651c2f32dbb928e9ed35"
server_key_iv: "83f760224f728af2eae03ed87654c7a5"
server_key_version: "encryptionkey@0"
I20260809 05:40:03.944672 27096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:03.961243 27096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:03.962265 27096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:03.962641 27096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:03.963824 27096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:03.965655 27096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:03.965804 27096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:03.965968 27096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:03.966071 27096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:04.005211 27096 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:45159
I20260809 05:40:04.005266 27289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:45159 every 8 connection(s)
I20260809 05:40:04.006844 27096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:40:04.016065 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 27096
I20260809 05:40:04.016270 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:40:04.016871 31271 external_mini_cluster.cc:1428] Setting key c94600ebf0224f360518f19302c3c71f
I20260809 05:40:04.020354 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42879
--builtin_ntp_servers=127.30.137.212:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:40:04.021123 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.009765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:44799 (local address 127.30.137.254:42879)
0809 05:40:04.009954 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.009959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.011412 (+ 1453us) server_negotiation.cc:407] Connection header received
0809 05:40:04.013238 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.013242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.013317 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.013414 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:04.014580 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.015418 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.017914 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.018395 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.018554 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.020243 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:04.020249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:04.020252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:04.020271 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:04.020305 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:04.020317 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:04.020391 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.020763 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.020878 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 05:40:04.022846 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.010219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42879 (local address 127.30.137.194:44799)
0809 05:40:04.011226 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.011266 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.012951 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.013638 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.013646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.013737 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:04.014395 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.014416 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.015658 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.015667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.017720 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.017736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.018686 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.018696 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.018946 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.019819 (+ 873us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:40:04.019823 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:04.019887 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:40:04.020009 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:04.020438 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:04.020450 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:04.020461 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.020737 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.021230 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":314,"threads_started":1}
I20260809 05:40:04.024614 27292 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42879
I20260809 05:40:04.024978 27292 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:04.025993 27292 heartbeater.cc:511] Master 127.30.137.254:42879 requested a full tablet report, sending...
I20260809 05:40:04.027902 26791 ts_manager.cc:194] Registered new tserver with Master: 18391f1408bd410a92f2c6b51b496234 (127.30.137.194:45159)
I20260809 05:40:04.029023 26791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.194:44799
W20260809 05:40:04.393533 27295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:04.393880 27295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:04.393971 27295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:04.401767 27295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:04.401888 27295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:04.402071 27295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:40:04.412078 27295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:42879
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:04.413394 27295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:04.415263 27295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:04.429134 27301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:04.429284 27302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:04.429244 27304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:04.430410 27295 server_base.cc:1034] running on GCE node
I20260809 05:40:04.431129 27295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:04.432163 27295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:04.433391 27295 hybrid_clock.cc:648] HybridClock initialized: now 1786254004433343 us; error 36 us; skew 500 ppm
I20260809 05:40:04.433702 27295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:04.439684 27295 webserver.cc:460] Webserver started at http://127.30.137.195:40487/ using document root <none> and password file <none>
I20260809 05:40:04.440461 27295 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:04.440564 27295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:04.440864 27295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:04.444267 27295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "f10cd14de6d84888bce608ca8a44dfb3"
format_stamp: "Formatted at 2026-08-09 05:40:04 on dist-test-slave-3kk6"
server_key: "f203c4093d78a8a8f653f350e1c4f5ce"
server_key_iv: "fac212c28ced3c47a5fe88656af40db7"
server_key_version: "encryptionkey@0"
I20260809 05:40:04.445250 27295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "f10cd14de6d84888bce608ca8a44dfb3"
format_stamp: "Formatted at 2026-08-09 05:40:04 on dist-test-slave-3kk6"
server_key: "f203c4093d78a8a8f653f350e1c4f5ce"
server_key_iv: "fac212c28ced3c47a5fe88656af40db7"
server_key_version: "encryptionkey@0"
I20260809 05:40:04.451310 27295 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 05:40:04.455137 27311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:04.456610 27295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 05:40:04.456823 27295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "f10cd14de6d84888bce608ca8a44dfb3"
format_stamp: "Formatted at 2026-08-09 05:40:04 on dist-test-slave-3kk6"
server_key: "f203c4093d78a8a8f653f350e1c4f5ce"
server_key_iv: "fac212c28ced3c47a5fe88656af40db7"
server_key_version: "encryptionkey@0"
I20260809 05:40:04.457088 27295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:04.475054 27295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:04.476037 27295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:04.476415 27295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:04.477530 27295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:04.479355 27295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:04.479507 27295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:04.479653 27295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:04.479750 27295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:04.519846 27295 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:45509
I20260809 05:40:04.519903 27467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:45509 every 8 connection(s)
I20260809 05:40:04.521437 27295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:40:04.527279 27092 heartbeater.cc:503] Master 127.30.137.254:42879 was elected leader, sending a full tablet report...
I20260809 05:40:04.529613 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 27295
I20260809 05:40:04.529817 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:40:04.530406 31271 external_mini_cluster.cc:1428] Setting key d829ee2317528282dc79d97acbeedfe4
I20260809 05:40:04.536046 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.524320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:53845 (local address 127.30.137.254:42879)
0809 05:40:04.524814 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.524819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.525926 (+ 1107us) server_negotiation.cc:407] Connection header received
0809 05:40:04.528280 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.528286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.528382 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.528515 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:04.529991 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.531078 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.532959 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.533407 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.533562 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.535081 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:04.535087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:04.535090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:04.535108 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:04.535151 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:04.535162 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:04.535281 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.535532 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.535640 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":270,"threads_started":1}
I20260809 05:40:04.537137 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.524761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:42879 (local address 127.30.137.195:53845)
0809 05:40:04.525721 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.525769 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.527948 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.528774 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.528784 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.528932 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:04.529764 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.529791 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.531263 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.531269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.532780 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.532793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.533662 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.533670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.533930 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.534717 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:40:04.534726 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:04.534773 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:40:04.534862 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:04.535282 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:04.535297 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:04.535307 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.535521 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.535968 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":294,"threads_started":1}
I20260809 05:40:04.538733 27468 heartbeater.cc:347] Connected to a master server at 127.30.137.254:42879
I20260809 05:40:04.539103 27468 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:04.540042 27468 heartbeater.cc:511] Master 127.30.137.254:42879 requested a full tablet report, sending...
I20260809 05:40:04.541721 26791 ts_manager.cc:194] Registered new tserver with Master: f10cd14de6d84888bce608ca8a44dfb3 (127.30.137.195:45509)
I20260809 05:40:04.542531 26791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.195:53845
I20260809 05:40:04.546007 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:40:04.557720 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.550807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.30.137.254:42879)
0809 05:40:04.551072 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.551082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.551618 (+ 536us) server_negotiation.cc:407] Connection header received
0809 05:40:04.551801 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.551805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.551894 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.551990 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:04.552530 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.553265 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.555004 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.555470 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.555583 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.556857 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:04.556863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:04.556866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:04.556887 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:04.556920 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:04.556932 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:04.557050 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.557245 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.557351 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 05:40:04.562454 26791 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:40:04.569859 26791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:40:04.606187 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.583635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:45159 (local address 127.0.0.1:59038)
0809 05:40:04.592218 (+ 8583us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.592297 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.592633 (+ 336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.596598 (+ 3965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.596611 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.596656 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:04.597401 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.597421 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.599923 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.599932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.603088 (+ 3156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.603106 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.604980 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.604986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.605214 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.605308 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.605482 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.605822 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8110,"mutex_wait_us":111,"thread_start_us":11619,"threads_started":1}
I20260809 05:40:04.606149 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.584065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59038 (local address 127.30.137.194:45159)
0809 05:40:04.595368 (+ 11303us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.595377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.595414 (+ 37us) server_negotiation.cc:407] Connection header received
0809 05:40:04.595528 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.595539 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.595962 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.596218 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:04.597681 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.599710 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.603358 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.604746 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.604962 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.605218 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.605522 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.605761 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11126,"thread_start_us":10990,"threads_started":1}
I20260809 05:40:04.607950 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.584725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.30.137.195:45509)
0809 05:40:04.594790 (+ 10065us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.594804 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.595681 (+ 877us) server_negotiation.cc:407] Connection header received
0809 05:40:04.595991 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.596003 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.596292 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.596488 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:04.599792 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.600865 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.604403 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.605991 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.606219 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.606644 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.607339 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.607529 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9747,"thread_start_us":10051,"threads_started":1}
I20260809 05:40:04.608786 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.583959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:45509 (local address 127.0.0.1:58428)
0809 05:40:04.595520 (+ 11561us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.595538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.595727 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.599001 (+ 3274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.599009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.599050 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:04.599531 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.599544 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.601028 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.601032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.604153 (+ 3121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.604170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.606816 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.606822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.607017 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.607056 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.607141 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.608399 (+ 1258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11445,"mutex_wait_us":46,"thread_start_us":6913,"threads_started":1}
I20260809 05:40:04.610929 27186 tablet_service.cc:1467] Processing CreateTablet for tablet 818c267951cf400fb9eb11ff5c816f13 (DEFAULT_TABLE table=test-table [id=61c7dade8946466f8b6a05e29fdcff50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:04.611644 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.595960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52690)
0809 05:40:04.601647 (+ 5687us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.601670 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.601825 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.602750 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.602756 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.602796 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:04.603428 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.603438 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.605108 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.605114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.608493 (+ 3379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.608509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.610854 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.610861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.611039 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.611071 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.611166 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.611303 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5538,"mutex_wait_us":4,"thread_start_us":5356,"threads_started":1}
I20260809 05:40:04.612596 27186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 818c267951cf400fb9eb11ff5c816f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:04.612677 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.596155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.30.137.193:39763)
0809 05:40:04.599895 (+ 3740us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.599904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.601979 (+ 2075us) server_negotiation.cc:407] Connection header received
0809 05:40:04.602100 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.602109 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.602481 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.602911 (+ 430us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:04.603600 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.604916 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.608739 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.610258 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.610487 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.610761 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.611284 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.612249 (+ 965us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3530,"thread_start_us":3376,"threads_started":1}
I20260809 05:40:04.612643 27373 tablet_service.cc:1467] Processing CreateTablet for tablet 818c267951cf400fb9eb11ff5c816f13 (DEFAULT_TABLE table=test-table [id=61c7dade8946466f8b6a05e29fdcff50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:04.614331 27373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 818c267951cf400fb9eb11ff5c816f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:04.616551 26997 tablet_service.cc:1467] Processing CreateTablet for tablet 818c267951cf400fb9eb11ff5c816f13 (DEFAULT_TABLE table=test-table [id=61c7dade8946466f8b6a05e29fdcff50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:04.618127 26997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 818c267951cf400fb9eb11ff5c816f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:04.628093 27499 tablet_bootstrap.cc:492] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Bootstrap starting.
I20260809 05:40:04.630165 27501 tablet_bootstrap.cc:492] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Bootstrap starting.
I20260809 05:40:04.631951 27499 tablet_bootstrap.cc:654] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:04.633394 27499 log.cc:824] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:04.633455 27504 tablet_bootstrap.cc:492] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Bootstrap starting.
I20260809 05:40:04.634749 27501 tablet_bootstrap.cc:654] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:04.636081 27501 log.cc:824] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:04.637486 27504 tablet_bootstrap.cc:654] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:04.638510 27504 log.cc:824] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:04.638584 27499 tablet_bootstrap.cc:492] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: No bootstrap required, opened a new log
I20260809 05:40:04.638934 27499 ts_tablet_manager.cc:1397] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260809 05:40:04.639539 27501 tablet_bootstrap.cc:492] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: No bootstrap required, opened a new log
I20260809 05:40:04.639861 27501 ts_tablet_manager.cc:1397] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260809 05:40:04.645450 27504 tablet_bootstrap.cc:492] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: No bootstrap required, opened a new log
I20260809 05:40:04.645763 27504 ts_tablet_manager.cc:1397] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260809 05:40:04.651167 27499 raft_consensus.cc:348] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.651832 27499 raft_consensus.cc:374] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:04.651947 27499 raft_consensus.cc:729] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18391f1408bd410a92f2c6b51b496234, State: Initialized, Role: FOLLOWER
I20260809 05:40:04.651737 27501 raft_consensus.cc:348] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.652226 27501 raft_consensus.cc:374] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:04.652331 27501 raft_consensus.cc:729] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f10cd14de6d84888bce608ca8a44dfb3, State: Initialized, Role: FOLLOWER
I20260809 05:40:04.652675 27499 consensus_queue.cc:260] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.652931 27501 consensus_queue.cc:260] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.654223 27468 heartbeater.cc:503] Master 127.30.137.254:42879 was elected leader, sending a full tablet report...
I20260809 05:40:04.655149 27504 raft_consensus.cc:348] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.655546 27504 raft_consensus.cc:374] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:04.655669 27504 raft_consensus.cc:729] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6329110dde1f4eeaa59ec4feb43eab2c, State: Initialized, Role: FOLLOWER
I20260809 05:40:04.656457 27504 consensus_queue.cc:260] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.657186 27501 ts_tablet_manager.cc:1428] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Time spent starting tablet: real 0.017s user 0.012s sys 0.002s
I20260809 05:40:04.658357 27292 heartbeater.cc:503] Master 127.30.137.254:42879 was elected leader, sending a full tablet report...
I20260809 05:40:04.659132 27499 ts_tablet_manager.cc:1428] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Time spent starting tablet: real 0.020s user 0.014s sys 0.006s
I20260809 05:40:04.660566 27504 ts_tablet_manager.cc:1428] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260809 05:40:04.671032 27511 raft_consensus.cc:482] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:40:04.671399 27511 raft_consensus.cc:504] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.673219 27511 leader_election.cc:288] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18391f1408bd410a92f2c6b51b496234 (127.30.137.194:45159), 6329110dde1f4eeaa59ec4feb43eab2c (127.30.137.193:39763)
I20260809 05:40:04.680512 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.673705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:46885 (local address 127.30.137.194:45159)
0809 05:40:04.673925 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.673930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.674042 (+ 112us) server_negotiation.cc:407] Connection header received
0809 05:40:04.674280 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.674287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.674378 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.674507 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:04.675171 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.676185 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.678499 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.679616 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.679743 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.679793 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.680185 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.680288 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:40:04.680660 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.673721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:45159 (local address 127.30.137.195:46885)
0809 05:40:04.673927 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.673951 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.674112 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.674573 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.674578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.674609 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:04.674962 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.674978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.676331 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.676343 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.678273 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.678289 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.679806 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.679812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.680000 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.680040 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.680155 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.680310 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":72}
I20260809 05:40:04.681571 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.674411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:59759 (local address 127.30.137.193:39763)
0809 05:40:04.674695 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.674704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.675333 (+ 629us) server_negotiation.cc:407] Connection header received
0809 05:40:04.675446 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.675453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.675543 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.675651 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:04.676185 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.676904 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.679350 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.680536 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.680702 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.680774 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.681267 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.681401 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:40:04.681674 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.674205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.30.137.195:59759)
0809 05:40:04.675224 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.675238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:04.675341 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:04.675674 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:04.675677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:04.675696 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:04.676002 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.676009 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.677068 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.677073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.679106 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:04.679121 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.680899 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.680905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:04.681076 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.681100 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 05:40:04.681176 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 05:40:04.681309 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":1,"thread_start_us":664,"threads_started":1}
I20260809 05:40:04.681929 27211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818c267951cf400fb9eb11ff5c816f13" candidate_uuid: "f10cd14de6d84888bce608ca8a44dfb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18391f1408bd410a92f2c6b51b496234" is_pre_election: true
I20260809 05:40:04.682578 27211 raft_consensus.cc:2454] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10cd14de6d84888bce608ca8a44dfb3 in term 0.
I20260809 05:40:04.682901 27023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818c267951cf400fb9eb11ff5c816f13" candidate_uuid: "f10cd14de6d84888bce608ca8a44dfb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" is_pre_election: true
I20260809 05:40:04.683439 27313 leader_election.cc:302] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18391f1408bd410a92f2c6b51b496234, f10cd14de6d84888bce608ca8a44dfb3; no voters:
I20260809 05:40:04.683516 27023 raft_consensus.cc:2454] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f10cd14de6d84888bce608ca8a44dfb3 in term 0.
I20260809 05:40:04.683951 27511 raft_consensus.cc:2781] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:40:04.684160 27511 raft_consensus.cc:482] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:40:04.684278 27511 raft_consensus.cc:3037] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:04.686810 27511 raft_consensus.cc:504] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.687496 27511 leader_election.cc:288] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [CANDIDATE]: Term 1 election: Requested vote from peers 18391f1408bd410a92f2c6b51b496234 (127.30.137.194:45159), 6329110dde1f4eeaa59ec4feb43eab2c (127.30.137.193:39763)
I20260809 05:40:04.688007 27211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818c267951cf400fb9eb11ff5c816f13" candidate_uuid: "f10cd14de6d84888bce608ca8a44dfb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18391f1408bd410a92f2c6b51b496234"
I20260809 05:40:04.688060 27023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "818c267951cf400fb9eb11ff5c816f13" candidate_uuid: "f10cd14de6d84888bce608ca8a44dfb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6329110dde1f4eeaa59ec4feb43eab2c"
I20260809 05:40:04.688248 27211 raft_consensus.cc:3037] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:04.688285 27023 raft_consensus.cc:3037] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:04.690600 27211 raft_consensus.cc:2454] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10cd14de6d84888bce608ca8a44dfb3 in term 1.
I20260809 05:40:04.690621 27023 raft_consensus.cc:2454] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f10cd14de6d84888bce608ca8a44dfb3 in term 1.
I20260809 05:40:04.691171 27313 leader_election.cc:302] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18391f1408bd410a92f2c6b51b496234, f10cd14de6d84888bce608ca8a44dfb3; no voters:
I20260809 05:40:04.691545 27511 raft_consensus.cc:2781] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:04.692113 27511 raft_consensus.cc:686] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 1 LEADER]: Becoming Leader. State: Replica: f10cd14de6d84888bce608ca8a44dfb3, State: Running, Role: LEADER
I20260809 05:40:04.692625 27511 consensus_queue.cc:237] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } }
I20260809 05:40:04.698004 26788 catalog_manager.cc:5495] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f10cd14de6d84888bce608ca8a44dfb3 (127.30.137.195). New cstate: current_term: 1 leader_uuid: "f10cd14de6d84888bce608ca8a44dfb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f10cd14de6d84888bce608ca8a44dfb3" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 45509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 } health_report { overall_health: UNKNOWN } } }
I20260809 05:40:04.736902 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.730861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.30.137.195:45509)
0809 05:40:04.731066 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.731074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.731100 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:40:04.731382 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.731386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.731468 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.731643 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:40:04.732111 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.732864 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.734396 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.734875 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.735045 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.735729 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:40:04.736254 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:40:04.736475 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.736596 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.736642 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:40:04.746110 27023 raft_consensus.cc:1261] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 1 FOLLOWER]: Refusing update from remote peer f10cd14de6d84888bce608ca8a44dfb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:04.746110 27211 raft_consensus.cc:1261] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 1 FOLLOWER]: Refusing update from remote peer f10cd14de6d84888bce608ca8a44dfb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:04.747272 27511 consensus_queue.cc:1035] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6329110dde1f4eeaa59ec4feb43eab2c" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 39763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:04.747673 27515 consensus_queue.cc:1035] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18391f1408bd410a92f2c6b51b496234" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 45159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 05:40:04.752460 27093 tablet.cc:2369] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:40:04.759361 27293 tablet.cc:2369] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:04.764627 27520 mvcc.cc:205] Tried to move back new op lower bound from 7316496403428208640 to 7316496403225100288. Current Snapshot: MvccSnapshot[applied={T|T < 7316496403428208640}]
I20260809 05:40:04.779709 26788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:04.780040 26788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:04.783743 26788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f800ce1c066249a584349233b41714f6: Sending DeleteTablet for 3 replicas of tablet 818c267951cf400fb9eb11ff5c816f13
I20260809 05:40:04.785086 27373 tablet_service.cc:1514] Processing DeleteTablet for tablet 818c267951cf400fb9eb11ff5c816f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:04 UTC) from {username='slave'} at 127.0.0.1:58428
I20260809 05:40:04.785143 27186 tablet_service.cc:1514] Processing DeleteTablet for tablet 818c267951cf400fb9eb11ff5c816f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:04 UTC) from {username='slave'} at 127.0.0.1:59038
I20260809 05:40:04.785190 26997 tablet_service.cc:1514] Processing DeleteTablet for tablet 818c267951cf400fb9eb11ff5c816f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:04 UTC) from {username='slave'} at 127.0.0.1:52690
I20260809 05:40:04.787750 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 26897
I20260809 05:40:04.799345 27530 tablet_replica.cc:330] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: stopping tablet replica
I20260809 05:40:04.799898 27530 raft_consensus.cc:2229] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:40:04.800657 27530 raft_consensus.cc:2258] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:04.802165 27532 tablet_replica.cc:330] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: stopping tablet replica
I20260809 05:40:04.803228 27530 ts_tablet_manager.cc:1905] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:04.808138 27532 raft_consensus.cc:2229] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:04.808733 27532 raft_consensus.cc:2258] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:04.809629 27533 tablet_replica.cc:330] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: stopping tablet replica
I20260809 05:40:04.810099 27533 raft_consensus.cc:2229] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:04.810608 27533 raft_consensus.cc:2258] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:04.811770 27530 ts_tablet_manager.cc:1918] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:04.811969 27530 log.cc:1196] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/818c267951cf400fb9eb11ff5c816f13
I20260809 05:40:04.812292 27532 ts_tablet_manager.cc:1905] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:04.812674 27530 ts_tablet_manager.cc:1939] T 818c267951cf400fb9eb11ff5c816f13 P f10cd14de6d84888bce608ca8a44dfb3: Deleting consensus metadata
I20260809 05:40:04.813391 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.789627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52694 (local address 127.30.137.193:39763)
0809 05:40:04.789892 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.789901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.798385 (+ 8484us) server_negotiation.cc:407] Connection header received
0809 05:40:04.798642 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.798667 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.798874 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.799025 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:04.802991 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.804696 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.807903 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.808893 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.809038 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.811486 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:04.811499 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:04.811518 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:04.812370 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:04.812441 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:04.812472 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:04.812597 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.812898 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.813048 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 05:40:04.814576 27087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:04.814937 26765 catalog_manager.cc:4841] TS f10cd14de6d84888bce608ca8a44dfb3 (127.30.137.195:45509): tablet 818c267951cf400fb9eb11ff5c816f13 (table test-table [id=61c7dade8946466f8b6a05e29fdcff50]) successfully deleted
I20260809 05:40:04.824098 27532 ts_tablet_manager.cc:1918] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:04.824307 27532 log.cc:1196] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/818c267951cf400fb9eb11ff5c816f13
I20260809 05:40:04.824960 27532 ts_tablet_manager.cc:1939] T 818c267951cf400fb9eb11ff5c816f13 P 18391f1408bd410a92f2c6b51b496234: Deleting consensus metadata
I20260809 05:40:04.827279 26766 catalog_manager.cc:4841] TS 18391f1408bd410a92f2c6b51b496234 (127.30.137.194:45159): tablet 818c267951cf400fb9eb11ff5c816f13 (table test-table [id=61c7dade8946466f8b6a05e29fdcff50]) successfully deleted
I20260809 05:40:04.816409 27533 ts_tablet_manager.cc:1905] T 818c267951cf400fb9eb11ff5c816f13 P 6329110dde1f4eeaa59ec4feb43eab2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:04.931123 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 26897
I20260809 05:40:04.961514 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 27096
W20260809 05:40:04.961481 26766 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.137.193:39763 (error 108)
I20260809 05:40:04.963857 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.963280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52704)
0809 05:40:04.963548 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:04.963676 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260809 05:40:04.964174 26766 catalog_manager.cc:4584] TS 6329110dde1f4eeaa59ec4feb43eab2c (127.30.137.193:39763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 818c267951cf400fb9eb11ff5c816f13: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
I20260809 05:40:04.969776 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:04.962903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.30.137.194:45159)
0809 05:40:04.963089 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:04.963095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:04.963114 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:40:04.963173 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:04.963177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:04.963260 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:04.963423 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:04.963892 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.964875 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.966265 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:04.966803 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:04.966960 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:04.968232 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:04.968242 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:04.968257 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:04.968900 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:04.968955 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:04.968977 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:04.969130 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:04.969269 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 05:40:04.969401 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:40:04.970620 27283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:05.018222 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.017686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52712)
0809 05:40:05.017947 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.018045 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 05:40:05.019486 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.019089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52726)
0809 05:40:05.019255 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.019348 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 05:40:05.080133 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.079599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52730)
0809 05:40:05.079850 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.079951 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 05:40:05.081306 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.080933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52744)
0809 05:40:05.081104 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.081190 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 05:40:05.123221 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 27096
I20260809 05:40:05.149207 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 27295
I20260809 05:40:05.157639 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.150130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.30.137.195:45509)
0809 05:40:05.150329 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:05.150334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:05.150432 (+ 98us) server_negotiation.cc:407] Connection header received
0809 05:40:05.150629 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:05.150634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:05.150835 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:05.150993 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:05.151628 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:05.152512 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:05.153926 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:05.154388 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:05.154575 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:05.156030 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:05.156039 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:05.156058 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:05.156741 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:05.156790 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:05.156811 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:05.156952 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:05.157101 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 05:40:05.157221 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 05:40:05.158466 27461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:05.162253 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.161765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52756)
0809 05:40:05.161942 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.162053 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 05:40:05.163787 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.163181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52770)
0809 05:40:05.163539 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.163630 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237}
I20260809 05:40:05.287309 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 27295
I20260809 05:40:05.314608 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 26745
I20260809 05:40:05.315929 26848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:05.429659 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:05.429107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:39763 (local address 127.0.0.1:52784)
0809 05:40:05.429332 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:05.429464 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:39763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 05:40:05.429857 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 26745
2026-08-09T05:40:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3177 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:05 dist-test-slave-3kk6 krb5kdc[27546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:05 dist-test-slave-3kk6 krb5kdc[27546](info): set up 2 sockets
Aug 09 05:40:05 dist-test-slave-3kk6 krb5kdc[27546](info): commencing operation
krb5kdc: starting...
W20260809 05:40:06.799984 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:07 dist-test-slave-3kk6 krb5kdc[27546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:07.164644 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:41141
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:41141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:07.529325 27561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:07.529691 27561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:07.529894 27561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:07.537528 27561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:07.537675 27561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:07.537801 27561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:07.537917 27561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:07.538030 27561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:07.549856 27561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:41141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:41141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:07.551493 27561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:07.553134 27561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:07.567080 27570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:07.566992 27573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:07.566968 27571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:07.568024 27561 server_base.cc:1034] running on GCE node
I20260809 05:40:07.568941 27561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:07.570525 27561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:07.571774 27561 hybrid_clock.cc:648] HybridClock initialized: now 1786254007571730 us; error 84 us; skew 500 ppm
I20260809 05:40:07.572161 27561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:07 dist-test-slave-3kk6 krb5kdc[27546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254007, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:07.575682 27561 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:07.577350 27561 webserver.cc:460] Webserver started at http://127.30.137.254:42915/ using document root <none> and password file <none>
I20260809 05:40:07.578104 27561 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:07.578262 27561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:07.578691 27561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:07.582273 27561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "3d8720b0acad42ec824badf60ff6c051"
format_stamp: "Formatted at 2026-08-09 05:40:07 on dist-test-slave-3kk6"
server_key: "9be303d03f1b7748d15e91de9eba9a90"
server_key_iv: "e23c52d8c156cbb6f17affaab26c47f6"
server_key_version: "encryptionkey@0"
I20260809 05:40:07.583170 27561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "3d8720b0acad42ec824badf60ff6c051"
format_stamp: "Formatted at 2026-08-09 05:40:07 on dist-test-slave-3kk6"
server_key: "9be303d03f1b7748d15e91de9eba9a90"
server_key_iv: "e23c52d8c156cbb6f17affaab26c47f6"
server_key_version: "encryptionkey@0"
I20260809 05:40:07.588922 27561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:07.592633 27582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:07.594048 27561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:40:07.594245 27561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "3d8720b0acad42ec824badf60ff6c051"
format_stamp: "Formatted at 2026-08-09 05:40:07 on dist-test-slave-3kk6"
server_key: "9be303d03f1b7748d15e91de9eba9a90"
server_key_iv: "e23c52d8c156cbb6f17affaab26c47f6"
server_key_version: "encryptionkey@0"
I20260809 05:40:07.594461 27561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:07.625027 27561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:07.629624 27561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:07.629968 27561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:07.651059 27561 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:41141
I20260809 05:40:07.651080 27657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:41141 every 8 connection(s)
I20260809 05:40:07.653018 27561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:07.657274 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:07.660377 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 27561
I20260809 05:40:07.660593 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:07.661098 31271 external_mini_cluster.cc:1428] Setting key b1c929fa15315d62fb74bbf4b490b0ba
I20260809 05:40:07.671182 27658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051: Bootstrap starting.
I20260809 05:40:07.675012 27658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:07.676112 27658 log.cc:824] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:40:07 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:07.679733 27658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051: No bootstrap required, opened a new log
I20260809 05:40:07.688396 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:07.663191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.30.137.254:41141)
0809 05:40:07.664044 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:07.664063 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:07.664114 (+ 51us) server_negotiation.cc:407] Connection header received
0809 05:40:07.666696 (+ 2582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:07.666733 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:07.667328 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:07.667842 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:07.669309 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:07.671374 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:07.672789 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:07.673888 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:07.674103 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:07.679090 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:07.679099 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:07.679116 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:07.679153 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:07.683369 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:07.684193 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:07.684203 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:07.684212 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:07.684444 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:07.684833 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:07.684839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:07.684842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:07.685620 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:07.685959 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:07.686319 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 05:40:07.686883 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":334,"threads_started":1}
I20260809 05:40:07.690632 27658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8720b0acad42ec824badf60ff6c051" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41141 } }
I20260809 05:40:07.691143 27658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:07.691277 27658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8720b0acad42ec824badf60ff6c051, State: Initialized, Role: FOLLOWER
I20260809 05:40:07.691933 27658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8720b0acad42ec824badf60ff6c051" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41141 } }
I20260809 05:40:07.692200 27658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:07.692349 27658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:07.692531 27658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:07.695052 27658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8720b0acad42ec824badf60ff6c051" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41141 } }
I20260809 05:40:07.695664 27658 leader_election.cc:302] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d8720b0acad42ec824badf60ff6c051; no voters:
I20260809 05:40:07.696328 27658 leader_election.cc:288] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:07.696480 27665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:07.697211 27665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [term 1 LEADER]: Becoming Leader. State: Replica: 3d8720b0acad42ec824badf60ff6c051, State: Running, Role: LEADER
I20260809 05:40:07.697943 27665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8720b0acad42ec824badf60ff6c051" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41141 } }
I20260809 05:40:07.698915 27658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:07.701676 27668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d8720b0acad42ec824badf60ff6c051. Latest consensus state: current_term: 1 leader_uuid: "3d8720b0acad42ec824badf60ff6c051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8720b0acad42ec824badf60ff6c051" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41141 } } }
I20260809 05:40:07.702203 27668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:07.702811 27666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d8720b0acad42ec824badf60ff6c051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8720b0acad42ec824badf60ff6c051" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41141 } } }
I20260809 05:40:07.703178 27666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:07.703923 27676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:07.709385 27676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:07.719300 27676 catalog_manager.cc:1347] Generated new cluster ID: 84437950b50349108f877f3eb2b54791
I20260809 05:40:07.719455 27676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:07.737952 27676 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:07.739482 27676 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:07.747221 27676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051: Generated new TSK 0
I20260809 05:40:07.748420 27676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.137.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
Entry for principal kudu/127.30.137.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab.
I20260809 05:40:07.836300 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:41141
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:08.202144 27690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:08.202477 27690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:08.202706 27690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:08.210860 27690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:08.210984 27690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:08.211187 27690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:40:08.221338 27690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.193.keytab
--principal=kudu/127.30.137.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:08.222819 27690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:08.224715 27690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:08.239353 27700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:08.239539 27698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:08.239506 27697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:08.240759 27690 server_base.cc:1034] running on GCE node
I20260809 05:40:08.241386 27690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:08.242362 27690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:08.243569 27690 hybrid_clock.cc:648] HybridClock initialized: now 1786254008243507 us; error 56 us; skew 500 ppm
I20260809 05:40:08.243872 27690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:08 dist-test-slave-3kk6 krb5kdc[27546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:08.247372 27690 init.cc:377] Logged in from keytab as kudu/127.30.137.193@KRBTEST.COM (short username kudu)
I20260809 05:40:08.248944 27690 webserver.cc:460] Webserver started at http://127.30.137.193:43111/ using document root <none> and password file <none>
I20260809 05:40:08.249656 27690 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:08.249780 27690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:08.250082 27690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:08.253381 27690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "31e41ad941ae42419cc0825766170b28"
format_stamp: "Formatted at 2026-08-09 05:40:08 on dist-test-slave-3kk6"
server_key: "8cdb72921ca333f62cd6d684288a76e3"
server_key_iv: "34a03ae6f4ef4cb67f378cecf7b6f7d5"
server_key_version: "encryptionkey@0"
I20260809 05:40:08.254238 27690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "31e41ad941ae42419cc0825766170b28"
format_stamp: "Formatted at 2026-08-09 05:40:08 on dist-test-slave-3kk6"
server_key: "8cdb72921ca333f62cd6d684288a76e3"
server_key_iv: "34a03ae6f4ef4cb67f378cecf7b6f7d5"
server_key_version: "encryptionkey@0"
I20260809 05:40:08.260347 27690 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 05:40:08.264297 27709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:08.265864 27690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 05:40:08.266063 27690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "31e41ad941ae42419cc0825766170b28"
format_stamp: "Formatted at 2026-08-09 05:40:08 on dist-test-slave-3kk6"
server_key: "8cdb72921ca333f62cd6d684288a76e3"
server_key_iv: "34a03ae6f4ef4cb67f378cecf7b6f7d5"
server_key_version: "encryptionkey@0"
I20260809 05:40:08.266299 27690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:08.303625 27690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:08.308332 27690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:08.308708 27690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:08.309839 27690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:08.311410 27690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:08.311494 27690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:08.311697 27690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:08.311759 27690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:08.351006 27690 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:38177
I20260809 05:40:08.351054 27890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:38177 every 8 connection(s)
I20260809 05:40:08.352566 27690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:40:08.353124 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 27690
I20260809 05:40:08.353379 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:40:08.354039 31271 external_mini_cluster.cc:1428] Setting key a6f158b8368919dc06fcfcae02a05cc9
Aug 09 05:40:08 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.193@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:08.377130 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:08.356305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:57911 (local address 127.30.137.254:41141)
0809 05:40:08.356845 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:08.356853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:08.358097 (+ 1244us) server_negotiation.cc:407] Connection header received
0809 05:40:08.360330 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:08.360346 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:08.360453 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:08.360594 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:08.362296 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.363524 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.365246 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.365980 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.366147 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:08.370147 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:08.370157 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:08.370164 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:08.370216 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:08.373469 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:08.374296 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:08.374305 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:08.374311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:08.374481 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:08.374974 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:08.374982 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:08.374988 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:08.375399 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:08.375563 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:08.376352 (+ 789us) server_negotiation.cc:299] Negotiation successful
0809 05:40:08.376640 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":277,"threads_started":1}
I20260809 05:40:08.377744 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:08.356997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:41141 (local address 127.30.137.193:57911)
0809 05:40:08.357893 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:08.357945 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:08.360001 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:08.360829 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:08.360839 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:08.361475 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:08.362096 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:08.362117 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.363721 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.363726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:08.365065 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:08.365080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.366242 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.366246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:08.366480 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:08.367388 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:08.367418 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:08.369780 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:08.373636 (+ 3856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:08.373644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:08.373658 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:08.374144 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:08.374601 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:08.374604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:08.374606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:08.374860 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:08.375535 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:08.375541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:08.375895 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 05:40:08.376136 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 05:40:08.376570 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":401,"threads_started":1}
I20260809 05:40:08.379475 27891 heartbeater.cc:347] Connected to a master server at 127.30.137.254:41141
I20260809 05:40:08.379814 27891 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:08.380577 27891 heartbeater.cc:511] Master 127.30.137.254:41141 requested a full tablet report, sending...
I20260809 05:40:08.382880 27611 ts_manager.cc:194] Registered new tserver with Master: 31e41ad941ae42419cc0825766170b28 (127.30.137.193:38177)
I20260809 05:40:08.384553 27611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.193@KRBTEST.COM'} at 127.30.137.193:57911
WARNING: no policy specified for kudu/127.30.137.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
Entry for principal kudu/127.30.137.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab.
I20260809 05:40:08.434309 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:41141
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:08.804389 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:08.804726 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:08.804910 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:08.812641 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:08.812788 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:08.812912 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:40:08.823057 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.194.keytab
--principal=kudu/127.30.137.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:08.824517 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:08.825978 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:08.840767 27906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:08.840878 27907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:08.840746 27909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:08.845292 27898 server_base.cc:1034] running on GCE node
I20260809 05:40:08.846371 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:08.847656 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:08.848937 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786254008848889 us; error 55 us; skew 500 ppm
I20260809 05:40:08.849354 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:08 dist-test-slave-3kk6 krb5kdc[27546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:08.853021 27898 init.cc:377] Logged in from keytab as kudu/127.30.137.194@KRBTEST.COM (short username kudu)
I20260809 05:40:08.854943 27898 webserver.cc:460] Webserver started at http://127.30.137.194:36147/ using document root <none> and password file <none>
I20260809 05:40:08.855824 27898 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:08.855984 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:08.856482 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:08.859933 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "460547a7708c48ea940ff0f3722ee8a7"
format_stamp: "Formatted at 2026-08-09 05:40:08 on dist-test-slave-3kk6"
server_key: "b769311c010bb89b3643df3caf057b61"
server_key_iv: "daa785211feb98f8b074e68139c25a03"
server_key_version: "encryptionkey@0"
I20260809 05:40:08.860829 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "460547a7708c48ea940ff0f3722ee8a7"
format_stamp: "Formatted at 2026-08-09 05:40:08 on dist-test-slave-3kk6"
server_key: "b769311c010bb89b3643df3caf057b61"
server_key_iv: "daa785211feb98f8b074e68139c25a03"
server_key_version: "encryptionkey@0"
I20260809 05:40:08.866711 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:40:08.870798 27917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:08.872365 27898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:08.872562 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "460547a7708c48ea940ff0f3722ee8a7"
format_stamp: "Formatted at 2026-08-09 05:40:08 on dist-test-slave-3kk6"
server_key: "b769311c010bb89b3643df3caf057b61"
server_key_iv: "daa785211feb98f8b074e68139c25a03"
server_key_version: "encryptionkey@0"
I20260809 05:40:08.872787 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:08.894033 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:08.899348 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:08.899698 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:08.900854 27898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:08.902441 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:08.902525 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:08.902735 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:08.902794 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:08.940142 27898 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:46803
I20260809 05:40:08.940232 28083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:46803 every 8 connection(s)
I20260809 05:40:08.941637 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:40:08.951510 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 27898
I20260809 05:40:08.951751 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:40:08.952204 31271 external_mini_cluster.cc:1428] Setting key 9d431b362b2192b11c69f516852f514b
Aug 09 05:40:08 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.194@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:08.964987 28086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:08.944641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:44967 (local address 127.30.137.254:41141)
0809 05:40:08.945152 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:08.945159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:08.946231 (+ 1072us) server_negotiation.cc:407] Connection header received
0809 05:40:08.948270 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:08.948276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:08.948355 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:08.948464 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:08.950180 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.951046 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.953135 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.953707 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.953871 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:08.958223 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:08.958229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:08.958232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:08.958270 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:08.961223 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:08.962120 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:08.962132 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:08.962134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:08.962285 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:08.962811 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:08.962816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:08.962824 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:08.963235 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:08.963397 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:08.964484 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0809 05:40:08.964683 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":372,"thread_start_us":231,"threads_started":1}
I20260809 05:40:08.966724 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:08.945265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:41141 (local address 127.30.137.194:44967)
0809 05:40:08.946041 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:08.946084 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:08.947960 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:08.948705 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:08.948713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:08.949332 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:08.949985 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:08.950006 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.951236 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.951240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:08.952880 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:08.952900 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:08.953990 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:08.953998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:08.954364 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:08.955228 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:08.955260 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:08.957917 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:08.961423 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:08.961434 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:08.961457 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:08.961975 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:08.962413 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:08.962418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:08.962422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:08.962641 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:08.963406 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:08.963417 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:08.964008 (+ 591us) client_negotiation.cc:769] Sending connection context
0809 05:40:08.964384 (+ 376us) client_negotiation.cc:240] Negotiation successful
0809 05:40:08.965064 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":289,"threads_started":1}
I20260809 05:40:08.968370 28084 heartbeater.cc:347] Connected to a master server at 127.30.137.254:41141
I20260809 05:40:08.968744 28084 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:08.969756 28084 heartbeater.cc:511] Master 127.30.137.254:41141 requested a full tablet report, sending...
I20260809 05:40:08.971865 27611 ts_manager.cc:194] Registered new tserver with Master: 460547a7708c48ea940ff0f3722ee8a7 (127.30.137.194:46803)
I20260809 05:40:08.972800 27611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.194@KRBTEST.COM'} at 127.30.137.194:44967
WARNING: no policy specified for kudu/127.30.137.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
Entry for principal kudu/127.30.137.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab.
I20260809 05:40:09.030404 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:41141
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:40:09.387732 27891 heartbeater.cc:503] Master 127.30.137.254:41141 was elected leader, sending a full tablet report...
W20260809 05:40:09.401089 28090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:09.401443 28090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:09.401626 28090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:09.409183 28090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:09.409339 28090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:09.409518 28090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:40:09.419799 28090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.195.keytab
--principal=kudu/127.30.137.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:41141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:09.421254 28090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:09.422750 28090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:09.437870 28097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:09.437865 28101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:09.437808 28099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:09.438619 28090 server_base.cc:1034] running on GCE node
I20260809 05:40:09.439422 28090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:09.440565 28090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:09.441779 28090 hybrid_clock.cc:648] HybridClock initialized: now 1786254009441733 us; error 41 us; skew 500 ppm
I20260809 05:40:09.442144 28090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:09 dist-test-slave-3kk6 krb5kdc[27546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:09.449817 28090 init.cc:377] Logged in from keytab as kudu/127.30.137.195@KRBTEST.COM (short username kudu)
I20260809 05:40:09.451598 28090 webserver.cc:460] Webserver started at http://127.30.137.195:36591/ using document root <none> and password file <none>
I20260809 05:40:09.452350 28090 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:09.452471 28090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:09.452764 28090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:09.456086 28090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "a80a2187f38d41a79f3e18125dcb3f68"
format_stamp: "Formatted at 2026-08-09 05:40:09 on dist-test-slave-3kk6"
server_key: "7a7d2d59b62a3f5a8c49ec92282ff1ae"
server_key_iv: "cba8f3f23765559895cb4cf3aedd850c"
server_key_version: "encryptionkey@0"
I20260809 05:40:09.456964 28090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "a80a2187f38d41a79f3e18125dcb3f68"
format_stamp: "Formatted at 2026-08-09 05:40:09 on dist-test-slave-3kk6"
server_key: "7a7d2d59b62a3f5a8c49ec92282ff1ae"
server_key_iv: "cba8f3f23765559895cb4cf3aedd850c"
server_key_version: "encryptionkey@0"
I20260809 05:40:09.462762 28090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:40:09.466516 28109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:09.467993 28090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 05:40:09.468288 28090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "a80a2187f38d41a79f3e18125dcb3f68"
format_stamp: "Formatted at 2026-08-09 05:40:09 on dist-test-slave-3kk6"
server_key: "7a7d2d59b62a3f5a8c49ec92282ff1ae"
server_key_iv: "cba8f3f23765559895cb4cf3aedd850c"
server_key_version: "encryptionkey@0"
I20260809 05:40:09.468519 28090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:09.512804 28090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:09.517597 28090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:09.517959 28090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:09.519062 28090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:09.520501 28090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:09.520589 28090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:09.520766 28090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:09.520819 28090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:09.556368 28090 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:39049
I20260809 05:40:09.556422 28273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:39049 every 8 connection(s)
I20260809 05:40:09.557801 28090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:40:09.567975 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 28090
I20260809 05:40:09.568202 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:40:09.568661 31271 external_mini_cluster.cc:1428] Setting key 505707739c001570a663c6b80205db84
Aug 09 05:40:09 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.195@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:09.580056 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.560816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:52325 (local address 127.30.137.254:41141)
0809 05:40:09.561312 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.561317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.562502 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 05:40:09.564578 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.564585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.564689 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.564819 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:09.566500 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.567398 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.569199 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.569676 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.569815 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.574130 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:09.574135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:09.574139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:09.574173 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:09.576889 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:09.577733 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:09.577737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:09.577740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:09.577884 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:09.578321 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:09.578324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:09.578327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:09.578609 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:09.578826 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.579533 (+ 707us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.579718 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":232,"threads_started":1}
I20260809 05:40:09.581601 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.561476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:41141 (local address 127.30.137.195:52325)
0809 05:40:09.562304 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:09.562362 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:09.564247 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:09.565091 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:09.565100 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:09.565701 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:09.566300 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.566318 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.567588 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.567592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.569043 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.569057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.570015 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.570027 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.570356 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.571299 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:09.571334 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:09.573905 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:09.577059 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:09.577067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:09.577081 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:09.577621 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:09.577998 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:09.578002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:09.578005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:09.578214 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:09.578819 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:09.578826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:09.579176 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 05:40:09.579440 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 05:40:09.579923 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":319,"threads_started":1}
I20260809 05:40:09.583370 28276 heartbeater.cc:347] Connected to a master server at 127.30.137.254:41141
I20260809 05:40:09.583710 28276 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:09.584610 28276 heartbeater.cc:511] Master 127.30.137.254:41141 requested a full tablet report, sending...
I20260809 05:40:09.586102 27611 ts_manager.cc:194] Registered new tserver with Master: a80a2187f38d41a79f3e18125dcb3f68 (127.30.137.195:39049)
I20260809 05:40:09.587001 27611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.137.195@KRBTEST.COM'} at 127.30.137.195:52325
I20260809 05:40:09.597688 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:40:09.614764 28278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.602259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.30.137.254:41141)
0809 05:40:09.602442 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.602446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.602868 (+ 422us) server_negotiation.cc:407] Connection header received
0809 05:40:09.603014 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.603017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.603083 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.603173 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:09.604626 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.605337 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.606614 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.607118 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.607221 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.609734 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:09.609739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:09.609742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:09.609774 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:09.612382 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:09.613028 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:09.613032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:09.613035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:09.613152 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:09.613480 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:09.613483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:09.613485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:09.613810 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:09.614019 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.614279 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.614458 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 05:40:09.618757 27611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:40:09.625890 27611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:40:09.658480 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.640136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.30.137.194:46803)
0809 05:40:09.649814 (+ 9678us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.649823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.649853 (+ 30us) server_negotiation.cc:407] Connection header received
0809 05:40:09.649960 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.649967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.650219 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.650472 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:09.651285 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.652952 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.656149 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.657199 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.657401 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.657624 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.657933 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.658118 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9466,"thread_start_us":9345,"threads_started":1}
I20260809 05:40:09.658567 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.640148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46803 (local address 127.0.0.1:53428)
0809 05:40:09.644986 (+ 4838us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:09.645013 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:09.645179 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:09.650490 (+ 5311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:09.650501 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:09.650539 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:09.651083 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.651096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.653135 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.653140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.655891 (+ 2751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.655907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.657440 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.657447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.657649 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.657710 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 05:40:09.657854 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 05:40:09.658093 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4656,"mutex_wait_us":58,"thread_start_us":483,"threads_started":1}
I20260809 05:40:09.661994 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.644112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:38177 (local address 127.0.0.1:47904)
0809 05:40:09.649082 (+ 4970us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:09.649103 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:09.649229 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:09.650140 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:09.650145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:09.650172 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:09.650829 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.650847 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.653134 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.653140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.657208 (+ 4068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.657222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.661086 (+ 3864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.661093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.661292 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.661329 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 05:40:09.661394 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 05:40:09.661545 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4850,"mutex_wait_us":23,"thread_start_us":942,"threads_started":1}
I20260809 05:40:09.663188 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.644886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.30.137.193:38177)
0809 05:40:09.649547 (+ 4661us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.649556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.649591 (+ 35us) server_negotiation.cc:407] Connection header received
0809 05:40:09.649694 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.649704 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.649929 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.650159 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:09.651125 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.652952 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.657435 (+ 4483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.659036 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.662340 (+ 3304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.662553 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.662629 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.662850 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4473,"thread_start_us":4318,"threads_started":1}
I20260809 05:40:09.663726 27995 tablet_service.cc:1467] Processing CreateTablet for tablet 02558e8880f84500baf9e8608f61024d (DEFAULT_TABLE table=test-table [id=d2450a418e204745a37a096395a0eb6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:09.664435 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.639652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:39049 (local address 127.0.0.1:37540)
0809 05:40:09.641163 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:09.641224 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:09.641481 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:09.649574 (+ 8093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:09.649582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:09.649615 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:09.650030 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.650044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.653013 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.653019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.655810 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.655827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.663568 (+ 7741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.663575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.663791 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.663826 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 05:40:09.663893 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 05:40:09.664036 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":71,"thread_start_us":9124,"threads_started":1}
I20260809 05:40:09.665187 27995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02558e8880f84500baf9e8608f61024d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:09.665675 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.639926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.30.137.195:39049)
0809 05:40:09.648687 (+ 8761us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.648700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.648742 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:40:09.648868 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.648879 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.649233 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.650800 (+ 1567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:09.651142 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.652836 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.659261 (+ 6425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.660809 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.664882 (+ 4073us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.665078 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.665147 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.665314 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8558,"thread_start_us":8399,"threads_started":1}
I20260809 05:40:09.669461 28183 tablet_service.cc:1467] Processing CreateTablet for tablet 02558e8880f84500baf9e8608f61024d (DEFAULT_TABLE table=test-table [id=d2450a418e204745a37a096395a0eb6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:09.670367 27782 tablet_service.cc:1467] Processing CreateTablet for tablet 02558e8880f84500baf9e8608f61024d (DEFAULT_TABLE table=test-table [id=d2450a418e204745a37a096395a0eb6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:09.671123 28183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02558e8880f84500baf9e8608f61024d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:09.671847 27782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02558e8880f84500baf9e8608f61024d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:09.684834 28299 tablet_bootstrap.cc:492] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Bootstrap starting.
I20260809 05:40:09.685698 28302 tablet_bootstrap.cc:492] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Bootstrap starting.
I20260809 05:40:09.687304 28304 tablet_bootstrap.cc:492] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Bootstrap starting.
I20260809 05:40:09.688889 28299 tablet_bootstrap.cc:654] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:09.689472 28302 tablet_bootstrap.cc:654] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:09.690044 28299 log.cc:824] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:09.690824 28302 log.cc:824] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:09.691138 28304 tablet_bootstrap.cc:654] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:09.692534 28304 log.cc:824] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:09.693814 28299 tablet_bootstrap.cc:492] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: No bootstrap required, opened a new log
I20260809 05:40:09.693928 28302 tablet_bootstrap.cc:492] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: No bootstrap required, opened a new log
I20260809 05:40:09.694151 28299 ts_tablet_manager.cc:1397] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260809 05:40:09.694229 28302 ts_tablet_manager.cc:1397] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 05:40:09.695798 28304 tablet_bootstrap.cc:492] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: No bootstrap required, opened a new log
I20260809 05:40:09.696111 28304 ts_tablet_manager.cc:1397] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260809 05:40:09.703186 28299 raft_consensus.cc:348] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.703660 28299 raft_consensus.cc:374] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:09.703806 28299 raft_consensus.cc:729] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460547a7708c48ea940ff0f3722ee8a7, State: Initialized, Role: FOLLOWER
I20260809 05:40:09.704514 28299 consensus_queue.cc:260] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.705799 28084 heartbeater.cc:503] Master 127.30.137.254:41141 was elected leader, sending a full tablet report...
I20260809 05:40:09.706349 28299 ts_tablet_manager.cc:1428] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
I20260809 05:40:09.706544 28302 raft_consensus.cc:348] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.707110 28302 raft_consensus.cc:374] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:09.707229 28302 raft_consensus.cc:729] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a80a2187f38d41a79f3e18125dcb3f68, State: Initialized, Role: FOLLOWER
I20260809 05:40:09.707270 28304 raft_consensus.cc:348] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.707839 28304 raft_consensus.cc:374] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:09.707959 28304 raft_consensus.cc:729] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e41ad941ae42419cc0825766170b28, State: Initialized, Role: FOLLOWER
I20260809 05:40:09.707914 28302 consensus_queue.cc:260] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.708573 28304 consensus_queue.cc:260] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.709204 28276 heartbeater.cc:503] Master 127.30.137.254:41141 was elected leader, sending a full tablet report...
I20260809 05:40:09.709899 28302 ts_tablet_manager.cc:1428] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260809 05:40:09.713536 28304 ts_tablet_manager.cc:1428] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260809 05:40:09.791229 28312 raft_consensus.cc:482] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:40:09.791579 28312 raft_consensus.cc:504] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.793598 28312 leader_election.cc:288] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a80a2187f38d41a79f3e18125dcb3f68 (127.30.137.195:39049), 460547a7708c48ea940ff0f3722ee8a7 (127.30.137.194:46803)
I20260809 05:40:09.802022 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.794127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:58847 (local address 127.30.137.195:39049)
0809 05:40:09.794381 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.794389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.795277 (+ 888us) server_negotiation.cc:407] Connection header received
0809 05:40:09.795525 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.795529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.795592 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.795691 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:09.796337 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.797076 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.799373 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.800687 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.800877 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.800954 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.801607 (+ 653us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.801740 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 05:40:09.802105 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.794427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:36435 (local address 127.30.137.194:46803)
0809 05:40:09.794730 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.794736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.795353 (+ 617us) server_negotiation.cc:407] Connection header received
0809 05:40:09.795554 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.795558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.795620 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.795715 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:09.796415 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.797139 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.799378 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.800700 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.800876 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.800954 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.801686 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.801835 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 05:40:09.802146 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.794118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:39049 (local address 127.30.137.193:58847)
0809 05:40:09.795132 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:09.795159 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:09.795365 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:09.795711 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:09.795715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:09.795749 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:09.796176 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.796191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.797212 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.797219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.799170 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.799187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.801243 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.801257 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.801426 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.801458 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 05:40:09.801575 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 05:40:09.801721 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":40,"thread_start_us":570,"threads_started":1}
I20260809 05:40:09.802924 28321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.794128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:46803 (local address 127.30.137.193:36435)
0809 05:40:09.795218 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:09.795233 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:09.795401 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:09.795745 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:09.795749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:09.795774 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:09.796218 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.796228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.797283 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.797289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.799170 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:09.799187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.801269 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.801274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:09.801475 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.801506 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:40:09.802377 (+ 871us) client_negotiation.cc:240] Negotiation successful
0809 05:40:09.802522 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":61,"spinlock_wait_cycles":24832,"thread_start_us":771,"threads_started":1}
I20260809 05:40:09.803537 28210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02558e8880f84500baf9e8608f61024d" candidate_uuid: "31e41ad941ae42419cc0825766170b28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a80a2187f38d41a79f3e18125dcb3f68" is_pre_election: true
I20260809 05:40:09.803994 28017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02558e8880f84500baf9e8608f61024d" candidate_uuid: "31e41ad941ae42419cc0825766170b28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460547a7708c48ea940ff0f3722ee8a7" is_pre_election: true
I20260809 05:40:09.804188 28210 raft_consensus.cc:2454] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31e41ad941ae42419cc0825766170b28 in term 0.
I20260809 05:40:09.804571 28017 raft_consensus.cc:2454] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31e41ad941ae42419cc0825766170b28 in term 0.
I20260809 05:40:09.805004 27715 leader_election.cc:302] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e41ad941ae42419cc0825766170b28, a80a2187f38d41a79f3e18125dcb3f68; no voters:
I20260809 05:40:09.805469 28312 raft_consensus.cc:2781] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:40:09.805641 28312 raft_consensus.cc:482] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:40:09.805758 28312 raft_consensus.cc:3037] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:09.808216 28312 raft_consensus.cc:504] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.808913 28312 leader_election.cc:288] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [CANDIDATE]: Term 1 election: Requested vote from peers a80a2187f38d41a79f3e18125dcb3f68 (127.30.137.195:39049), 460547a7708c48ea940ff0f3722ee8a7 (127.30.137.194:46803)
I20260809 05:40:09.809478 28017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02558e8880f84500baf9e8608f61024d" candidate_uuid: "31e41ad941ae42419cc0825766170b28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460547a7708c48ea940ff0f3722ee8a7"
I20260809 05:40:09.809528 28210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02558e8880f84500baf9e8608f61024d" candidate_uuid: "31e41ad941ae42419cc0825766170b28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a80a2187f38d41a79f3e18125dcb3f68"
W20260809 05:40:09.809578 28277 tablet.cc:2369] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:09.809731 28017 raft_consensus.cc:3037] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:09.809751 28210 raft_consensus.cc:3037] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:09.813220 28017 raft_consensus.cc:2454] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31e41ad941ae42419cc0825766170b28 in term 1.
I20260809 05:40:09.813230 28210 raft_consensus.cc:2454] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31e41ad941ae42419cc0825766170b28 in term 1.
I20260809 05:40:09.813773 27710 leader_election.cc:302] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 31e41ad941ae42419cc0825766170b28, 460547a7708c48ea940ff0f3722ee8a7, a80a2187f38d41a79f3e18125dcb3f68; no voters:
I20260809 05:40:09.814150 28312 raft_consensus.cc:2781] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:09.814638 28312 raft_consensus.cc:686] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 1 LEADER]: Becoming Leader. State: Replica: 31e41ad941ae42419cc0825766170b28, State: Running, Role: LEADER
I20260809 05:40:09.815186 28312 consensus_queue.cc:237] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } }
I20260809 05:40:09.820007 27610 catalog_manager.cc:5495] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31e41ad941ae42419cc0825766170b28 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "31e41ad941ae42419cc0825766170b28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31e41ad941ae42419cc0825766170b28" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 38177 } health_report { overall_health: HEALTHY } } }
I20260809 05:40:09.852545 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.846817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.30.137.193:38177)
0809 05:40:09.847050 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.847056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.847079 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:40:09.847237 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.847241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.847325 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.847492 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:40:09.847903 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.848658 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.850126 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.850561 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.850730 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.851352 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:40:09.851915 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:40:09.852089 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.852149 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.852246 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 05:40:09.856508 27892 tablet.cc:2369] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:09.861508 28017 raft_consensus.cc:1261] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 1 FOLLOWER]: Refusing update from remote peer 31e41ad941ae42419cc0825766170b28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:09.861507 28210 raft_consensus.cc:1261] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 1 FOLLOWER]: Refusing update from remote peer 31e41ad941ae42419cc0825766170b28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:09.862550 28323 consensus_queue.cc:1035] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460547a7708c48ea940ff0f3722ee8a7" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 46803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:09.863006 28312 consensus_queue.cc:1035] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a80a2187f38d41a79f3e18125dcb3f68" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:09.875705 28327 mvcc.cc:205] Tried to move back new op lower bound from 7316496424381591552 to 7316496424207044608. Current Snapshot: MvccSnapshot[applied={T|T < 7316496424381591552}]
I20260809 05:40:09.891955 27611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:09.892288 27611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:09.896302 27611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d8720b0acad42ec824badf60ff6c051: Sending DeleteTablet for 3 replicas of tablet 02558e8880f84500baf9e8608f61024d
I20260809 05:40:09.897789 27782 tablet_service.cc:1514] Processing DeleteTablet for tablet 02558e8880f84500baf9e8608f61024d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:09 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:47904
I20260809 05:40:09.897799 27995 tablet_service.cc:1514] Processing DeleteTablet for tablet 02558e8880f84500baf9e8608f61024d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:09 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:53428
I20260809 05:40:09.898041 28183 tablet_service.cc:1514] Processing DeleteTablet for tablet 02558e8880f84500baf9e8608f61024d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:09 UTC) from {username='kudu', principal='kudu/127.30.137.254@KRBTEST.COM'} at 127.0.0.1:37540
I20260809 05:40:09.900185 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 27690
I20260809 05:40:09.913992 28341 tablet_replica.cc:330] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: stopping tablet replica
I20260809 05:40:09.914999 28341 raft_consensus.cc:2229] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:09.915912 28341 raft_consensus.cc:2258] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:09.916698 28340 tablet_replica.cc:330] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: stopping tablet replica
I20260809 05:40:09.916961 28339 tablet_replica.cc:330] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: stopping tablet replica
I20260809 05:40:09.917477 28339 raft_consensus.cc:2229] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:09.917559 28340 raft_consensus.cc:2229] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:40:09.918116 28339 raft_consensus.cc:2258] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:09.920506 28341 ts_tablet_manager.cc:1905] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:09.920648 28339 ts_tablet_manager.cc:1905] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:09.920917 28340 raft_consensus.cc:2258] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:09.923552 28340 ts_tablet_manager.cc:1905] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 05:40:09 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.193@KRBTEST.COM
I20260809 05:40:09.928483 28341 ts_tablet_manager.cc:1918] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:09.928484 28339 ts_tablet_manager.cc:1918] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:09.928684 28339 log.cc:1196] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/02558e8880f84500baf9e8608f61024d
I20260809 05:40:09.928853 28341 log.cc:1196] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/02558e8880f84500baf9e8608f61024d
I20260809 05:40:09.929360 28339 ts_tablet_manager.cc:1939] T 02558e8880f84500baf9e8608f61024d P 460547a7708c48ea940ff0f3722ee8a7: Deleting consensus metadata
I20260809 05:40:09.929563 28341 ts_tablet_manager.cc:1939] T 02558e8880f84500baf9e8608f61024d P a80a2187f38d41a79f3e18125dcb3f68: Deleting consensus metadata
I20260809 05:40:09.931634 27586 catalog_manager.cc:4841] TS a80a2187f38d41a79f3e18125dcb3f68 (127.30.137.195:39049): tablet 02558e8880f84500baf9e8608f61024d (table test-table [id=d2450a418e204745a37a096395a0eb6d]) successfully deleted
I20260809 05:40:09.932039 27583 catalog_manager.cc:4841] TS 460547a7708c48ea940ff0f3722ee8a7 (127.30.137.194:46803): tablet 02558e8880f84500baf9e8608f61024d (table test-table [id=d2450a418e204745a37a096395a0eb6d]) successfully deleted
I20260809 05:40:09.933799 28340 ts_tablet_manager.cc:1918] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:09.934013 28340 log.cc:1196] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786253675866932-31271-0/minicluster-data/ts-0/wal/wals/02558e8880f84500baf9e8608f61024d
I20260809 05:40:09.934711 28340 ts_tablet_manager.cc:1939] T 02558e8880f84500baf9e8608f61024d P 31e41ad941ae42419cc0825766170b28: Deleting consensus metadata
I20260809 05:40:09.936863 27585 catalog_manager.cc:4841] TS 31e41ad941ae42419cc0825766170b28 (127.30.137.193:38177): tablet 02558e8880f84500baf9e8608f61024d (table test-table [id=d2450a418e204745a37a096395a0eb6d]) successfully deleted
I20260809 05:40:09.940001 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:09.901158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.30.137.193:38177)
0809 05:40:09.901369 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:09.901376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:09.906759 (+ 5383us) server_negotiation.cc:407] Connection header received
0809 05:40:09.907864 (+ 1105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:09.907871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:09.907973 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:09.908058 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:09.911968 (+ 3910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.913287 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.916802 (+ 3515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:09.917548 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:09.917678 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:09.932654 (+ 14976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:09.932667 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:09.932682 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:09.932720 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:09.936368 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:09.937372 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:09.937381 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:09.937383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:09.937554 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:09.938013 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:09.938017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:09.938020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:09.938415 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:09.938781 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:09.939138 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 05:40:09.939419 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:40:09.942063 27884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:10.047309 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 27690
I20260809 05:40:10.076277 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 27898
Aug 09 05:40:10 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.194@KRBTEST.COM
I20260809 05:40:10.093179 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:10.077133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.30.137.194:46803)
0809 05:40:10.077366 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:10.077373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:10.077396 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:40:10.077611 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:10.077616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:10.077684 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:10.077800 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:10.079223 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:10.080002 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:10.081347 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:10.081759 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:10.081909 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:10.086771 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:10.086789 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:10.086806 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:10.086854 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:10.090015 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:10.090867 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:10.090875 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:10.090878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:10.091018 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:10.091461 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:10.091466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:10.091468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:10.091909 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:10.092287 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:10.092541 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 05:40:10.092782 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:40:10.093948 28076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:10.194226 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 27898
I20260809 05:40:10.217093 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 28090
Aug 09 05:40:10 dist-test-slave-3kk6 krb5kdc[27546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.195@KRBTEST.COM
I20260809 05:40:10.235694 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:10.218044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37544 (local address 127.30.137.195:39049)
0809 05:40:10.218244 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:10.218249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:10.218270 (+ 21us) server_negotiation.cc:407] Connection header received
0809 05:40:10.218441 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:10.218445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:10.218513 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:10.218603 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:10.220290 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:10.221043 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:10.222436 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:10.223019 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:10.223124 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:10.228138 (+ 5014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:10.228158 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:10.228175 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:10.228231 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:10.231900 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:10.233149 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:10.233161 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:10.233165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:10.233367 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:10.233817 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:10.233823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:10.233834 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:10.234363 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:10.234718 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:10.235063 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 05:40:10.235313 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:40:10.236398 28266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:10.327611 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 28090
I20260809 05:40:10.349846 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 27561
I20260809 05:40:10.351150 27653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:10.466130 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 27561
2026-08-09T05:40:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5040 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T05:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:10Z Disabled control of system clock
I20260809 05:40:10.576882 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:45729
--webserver_interface=127.30.137.254
--webserver_port=0
--builtin_ntp_servers=127.30.137.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:45729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:40:10.935401 28358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:10.935740 28358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:10.935829 28358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:10.943296 28358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:10.943432 28358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:10.943472 28358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:10.943538 28358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:10.943585 28358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:10.953428 28358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:45729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:45729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:10.954885 28358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:10.956682 28358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:10.969471 28366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:10.969544 28363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:10.969540 28364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:10.970966 28358 server_base.cc:1034] running on GCE node
I20260809 05:40:10.971915 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:10.973516 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:10.974781 28358 hybrid_clock.cc:648] HybridClock initialized: now 1786254010974715 us; error 57 us; skew 500 ppm
I20260809 05:40:10.975181 28358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:10.977663 28358 webserver.cc:460] Webserver started at http://127.30.137.254:44079/ using document root <none> and password file <none>
I20260809 05:40:10.978430 28358 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:10.978571 28358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:10.978947 28358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:10.982153 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "896c88eeb7844b958ee570dbfc18ce5b"
format_stamp: "Formatted at 2026-08-09 05:40:10 on dist-test-slave-3kk6"
server_key: "80bf6a42e53f73bbf225d4b6d1ca83b6"
server_key_iv: "1c624dc12346939279c6e99d9b840a2d"
server_key_version: "encryptionkey@0"
I20260809 05:40:10.983103 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "896c88eeb7844b958ee570dbfc18ce5b"
format_stamp: "Formatted at 2026-08-09 05:40:10 on dist-test-slave-3kk6"
server_key: "80bf6a42e53f73bbf225d4b6d1ca83b6"
server_key_iv: "1c624dc12346939279c6e99d9b840a2d"
server_key_version: "encryptionkey@0"
I20260809 05:40:10.988691 28358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:10.992503 28375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:10.993989 28358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:40:10.994202 28358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "896c88eeb7844b958ee570dbfc18ce5b"
format_stamp: "Formatted at 2026-08-09 05:40:10 on dist-test-slave-3kk6"
server_key: "80bf6a42e53f73bbf225d4b6d1ca83b6"
server_key_iv: "1c624dc12346939279c6e99d9b840a2d"
server_key_version: "encryptionkey@0"
I20260809 05:40:10.994442 28358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:11.035739 28358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:11.036712 28358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:11.037046 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:11.057135 28358 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:45729
I20260809 05:40:11.057158 28443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:45729 every 8 connection(s)
I20260809 05:40:11.058676 28358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:11.060976 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 28358
I20260809 05:40:11.061178 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:11.061664 31271 external_mini_cluster.cc:1428] Setting key aa954068cf155991d80ffe9cfbe0a99c
I20260809 05:40:11.063078 28444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:11.078447 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:11.063809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.30.137.254:45729)
0809 05:40:11.064976 (+ 1167us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:11.064993 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:11.065051 (+ 58us) server_negotiation.cc:407] Connection header received
0809 05:40:11.067036 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:11.067067 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:11.067747 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:11.068281 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:11.068855 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:11.071029 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:11.072589 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:11.073682 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:11.073905 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:11.075246 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:11.075257 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:11.075279 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:11.076116 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:11.076227 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:11.076260 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:11.076395 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:11.076644 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 05:40:11.077054 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":381,"threads_started":1}
I20260809 05:40:11.079463 28444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b: Bootstrap starting.
I20260809 05:40:11.082644 28444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:11.083755 28444 log.cc:824] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:11.086711 28444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b: No bootstrap required, opened a new log
I20260809 05:40:11.094587 28444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896c88eeb7844b958ee570dbfc18ce5b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45729 } }
I20260809 05:40:11.095208 28444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:11.095352 28444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896c88eeb7844b958ee570dbfc18ce5b, State: Initialized, Role: FOLLOWER
I20260809 05:40:11.095952 28444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896c88eeb7844b958ee570dbfc18ce5b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45729 } }
I20260809 05:40:11.096208 28444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:11.096379 28444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:11.096556 28444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:11.099105 28444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896c88eeb7844b958ee570dbfc18ce5b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45729 } }
I20260809 05:40:11.099655 28444 leader_election.cc:302] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 896c88eeb7844b958ee570dbfc18ce5b; no voters:
I20260809 05:40:11.100306 28444 leader_election.cc:288] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:11.100471 28449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:11.101799 28449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [term 1 LEADER]: Becoming Leader. State: Replica: 896c88eeb7844b958ee570dbfc18ce5b, State: Running, Role: LEADER
I20260809 05:40:11.102398 28449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896c88eeb7844b958ee570dbfc18ce5b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45729 } }
I20260809 05:40:11.102797 28444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:11.105741 28453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 896c88eeb7844b958ee570dbfc18ce5b. Latest consensus state: current_term: 1 leader_uuid: "896c88eeb7844b958ee570dbfc18ce5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896c88eeb7844b958ee570dbfc18ce5b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45729 } } }
I20260809 05:40:11.106184 28453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:11.108007 28460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:11.107916 28450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "896c88eeb7844b958ee570dbfc18ce5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "896c88eeb7844b958ee570dbfc18ce5b" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 45729 } } }
I20260809 05:40:11.108237 28450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:11.113058 28460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:11.122941 28460 catalog_manager.cc:1347] Generated new cluster ID: 015162ba8076410bac4a8747090cbcae
I20260809 05:40:11.123092 28460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:11.135221 28460 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:11.136606 28460 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:11.145489 28460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b: Generated new TSK 0
I20260809 05:40:11.146502 28460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:11.153920 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.193:0
--local_ip_for_outbound_sockets=127.30.137.193
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45729
--builtin_ntp_servers=127.30.137.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 05:40:11.522588 28474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:11.522994 28474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:11.523138 28474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:11.530799 28474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:11.530921 28474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:11.531117 28474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.193
I20260809 05:40:11.540760 28474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.137.193
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.193
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:11.542160 28474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:11.544178 28474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:11.558470 28485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:11.558621 28484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:11.558493 28488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:11.559541 28474 server_base.cc:1034] running on GCE node
I20260809 05:40:11.560230 28474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:11.561215 28474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:11.562492 28474 hybrid_clock.cc:648] HybridClock initialized: now 1786254011562448 us; error 54 us; skew 500 ppm
I20260809 05:40:11.562918 28474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:11.565510 28474 webserver.cc:460] Webserver started at http://127.30.137.193:42371/ using document root <none> and password file <none>
I20260809 05:40:11.566263 28474 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:11.566408 28474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:11.566846 28474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:11.570091 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data/instance:
uuid: "de64a53803ef46cb92d917bfce2c8db9"
format_stamp: "Formatted at 2026-08-09 05:40:11 on dist-test-slave-3kk6"
server_key: "29959a61a0ead8c255974f6cd1e168ed"
server_key_iv: "166981668ce43d6b51a1f482d8efefe5"
server_key_version: "encryptionkey@0"
I20260809 05:40:11.571141 28474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance:
uuid: "de64a53803ef46cb92d917bfce2c8db9"
format_stamp: "Formatted at 2026-08-09 05:40:11 on dist-test-slave-3kk6"
server_key: "29959a61a0ead8c255974f6cd1e168ed"
server_key_iv: "166981668ce43d6b51a1f482d8efefe5"
server_key_version: "encryptionkey@0"
I20260809 05:40:11.577239 28474 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 05:40:11.581166 28497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:11.582629 28474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 05:40:11.582882 28474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal
uuid: "de64a53803ef46cb92d917bfce2c8db9"
format_stamp: "Formatted at 2026-08-09 05:40:11 on dist-test-slave-3kk6"
server_key: "29959a61a0ead8c255974f6cd1e168ed"
server_key_iv: "166981668ce43d6b51a1f482d8efefe5"
server_key_version: "encryptionkey@0"
I20260809 05:40:11.583129 28474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:11.597131 28474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:11.598179 28474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:11.598516 28474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:11.599704 28474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:11.601284 28474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:11.601413 28474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:11.601540 28474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:11.601634 28474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:11.638546 28474 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.193:44731
I20260809 05:40:11.638593 28649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.193:44731 every 8 connection(s)
I20260809 05:40:11.640833 28474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/data/info.pb
I20260809 05:40:11.650712 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 28474
I20260809 05:40:11.650947 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-0/wal/instance
I20260809 05:40:11.651425 31271 external_mini_cluster.cc:1428] Setting key 03bfb04b8ac0f2e87fbd6546fbcb42c7
I20260809 05:40:11.654618 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.194:0
--local_ip_for_outbound_sockets=127.30.137.194
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45729
--builtin_ntp_servers=127.30.137.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:40:11.658344 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:11.646263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:40265 (local address 127.30.137.254:45729)
0809 05:40:11.646823 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:11.646830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:11.648140 (+ 1310us) server_negotiation.cc:407] Connection header received
0809 05:40:11.650301 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:11.650310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:11.650442 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:11.650590 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:11.651996 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:11.652849 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:11.654872 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:11.655626 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:11.655793 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:11.657277 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:11.657286 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:11.657294 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:11.657323 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:11.657390 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:11.657409 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:11.657507 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:11.657743 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 05:40:11.657867 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":249,"threads_started":1}
I20260809 05:40:11.659304 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:11.646736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45729 (local address 127.30.137.193:40265)
0809 05:40:11.647930 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:11.647987 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:11.649942 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:11.650937 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:11.650948 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:11.651076 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:11.651780 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:11.651801 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:11.653037 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:11.653042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:11.654631 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:11.654698 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:11.655873 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:11.655878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:11.656098 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:11.656891 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:40:11.656894 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:11.656940 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:40:11.657033 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:11.657514 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:11.657524 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:11.657530 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:40:11.657682 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 05:40:11.658034 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":383,"threads_started":1}
I20260809 05:40:11.661039 28650 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45729
I20260809 05:40:11.661412 28650 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:11.662417 28650 heartbeater.cc:511] Master 127.30.137.254:45729 requested a full tablet report, sending...
I20260809 05:40:11.664917 28397 ts_manager.cc:194] Registered new tserver with Master: de64a53803ef46cb92d917bfce2c8db9 (127.30.137.193:44731)
I20260809 05:40:11.666939 28397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.193:40265
W20260809 05:40:12.022868 28654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:12.023203 28654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:12.023384 28654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:12.031059 28654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:12.031190 28654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:12.031404 28654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.194
I20260809 05:40:12.040987 28654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.137.194
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.194
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:12.042423 28654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:12.044435 28654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:12.058343 28661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:12.058996 28663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:12.059114 28660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:12.059782 28654 server_base.cc:1034] running on GCE node
I20260809 05:40:12.060536 28654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:12.061599 28654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:12.062827 28654 hybrid_clock.cc:648] HybridClock initialized: now 1786254012062770 us; error 53 us; skew 500 ppm
I20260809 05:40:12.063169 28654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:12.070910 28654 webserver.cc:460] Webserver started at http://127.30.137.194:37377/ using document root <none> and password file <none>
I20260809 05:40:12.071904 28654 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:12.072057 28654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:12.072484 28654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:12.075829 28654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data/instance:
uuid: "a98e3aabf0444231a36fc3ef16263577"
format_stamp: "Formatted at 2026-08-09 05:40:12 on dist-test-slave-3kk6"
server_key: "61edfb80e6be2d1cf4fe42f5f701e0db"
server_key_iv: "aeccf856eb24ceec2c46e12b62a46717"
server_key_version: "encryptionkey@0"
I20260809 05:40:12.076725 28654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance:
uuid: "a98e3aabf0444231a36fc3ef16263577"
format_stamp: "Formatted at 2026-08-09 05:40:12 on dist-test-slave-3kk6"
server_key: "61edfb80e6be2d1cf4fe42f5f701e0db"
server_key_iv: "aeccf856eb24ceec2c46e12b62a46717"
server_key_version: "encryptionkey@0"
I20260809 05:40:12.082561 28654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 05:40:12.086304 28670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:12.087906 28654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 05:40:12.088114 28654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal
uuid: "a98e3aabf0444231a36fc3ef16263577"
format_stamp: "Formatted at 2026-08-09 05:40:12 on dist-test-slave-3kk6"
server_key: "61edfb80e6be2d1cf4fe42f5f701e0db"
server_key_iv: "aeccf856eb24ceec2c46e12b62a46717"
server_key_version: "encryptionkey@0"
I20260809 05:40:12.088376 28654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:12.118633 28654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:12.119632 28654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:12.119972 28654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:12.121106 28654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:12.122711 28654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:12.122834 28654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:12.122967 28654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:12.123068 28654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:12.158934 28654 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.194:40177
I20260809 05:40:12.158981 28826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.194:40177 every 8 connection(s)
I20260809 05:40:12.160651 28654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/data/info.pb
I20260809 05:40:12.163113 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 28654
I20260809 05:40:12.163341 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal/instance
I20260809 05:40:12.163923 31271 external_mini_cluster.cc:1428] Setting key 4bc7d1aacc940736ded468dfdd2bcaf1
I20260809 05:40:12.167117 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.137.195:0
--local_ip_for_outbound_sockets=127.30.137.195
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45729
--builtin_ntp_servers=127.30.137.212:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 05:40:12.174853 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.163658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.194:53957 (local address 127.30.137.254:45729)
0809 05:40:12.164165 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.164170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.166744 (+ 2574us) server_negotiation.cc:407] Connection header received
0809 05:40:12.167924 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.167930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:12.168030 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:12.168145 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:12.169292 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.170080 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.171785 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.172231 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.172375 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.173909 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:12.173914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:12.173917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:12.173938 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:12.173979 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:12.173991 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:12.174089 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:12.174326 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 05:40:12.174439 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":270,"threads_started":1}
I20260809 05:40:12.175917 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.164392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45729 (local address 127.30.137.194:53957)
0809 05:40:12.165602 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:12.165659 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:12.167633 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:12.168415 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:12.168424 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:12.168524 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:12.169117 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.169133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.170247 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.170252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.171621 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.171634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.172510 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.172521 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.172750 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.173538 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:40:12.173541 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:12.173586 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:40:12.173678 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:12.174091 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:12.174106 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:12.174117 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 05:40:12.174314 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 05:40:12.174802 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":380,"threads_started":1}
I20260809 05:40:12.177624 28827 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45729
I20260809 05:40:12.177948 28827 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:12.178797 28827 heartbeater.cc:511] Master 127.30.137.254:45729 requested a full tablet report, sending...
I20260809 05:40:12.180456 28397 ts_manager.cc:194] Registered new tserver with Master: a98e3aabf0444231a36fc3ef16263577 (127.30.137.194:40177)
I20260809 05:40:12.181716 28397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.194:53957
W20260809 05:40:12.548867 28831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:12.549252 28831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:12.549427 28831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:12.557160 28831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:12.557351 28831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:12.557528 28831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.137.195
I20260809 05:40:12.567368 28831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.137.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.137.195
--webserver_port=0
--tserver_master_addrs=127.30.137.254:45729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.137.195
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:12.568769 28831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:12.570770 28831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:12.584823 28839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:12.584877 28841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:12.584908 28843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:12.585896 28831 server_base.cc:1034] running on GCE node
I20260809 05:40:12.586599 28831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:12.587764 28831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:12.589010 28831 hybrid_clock.cc:648] HybridClock initialized: now 1786254012588961 us; error 45 us; skew 500 ppm
I20260809 05:40:12.589356 28831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 05:40:12.591778 28831 webserver.cc:460] Webserver started at http://127.30.137.195:38559/ using document root <none> and password file <none>
I20260809 05:40:12.592567 28831 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:12.592702 28831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:12.593035 28831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:12.596170 28831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data/instance:
uuid: "2813db21c200419aa7a061f5419bff26"
format_stamp: "Formatted at 2026-08-09 05:40:12 on dist-test-slave-3kk6"
server_key: "f463cff939cdbc4ff5d7f9b8ed57bd13"
server_key_iv: "d71269b190a73bbe9dde63d16e9a587b"
server_key_version: "encryptionkey@0"
I20260809 05:40:12.597011 28831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance:
uuid: "2813db21c200419aa7a061f5419bff26"
format_stamp: "Formatted at 2026-08-09 05:40:12 on dist-test-slave-3kk6"
server_key: "f463cff939cdbc4ff5d7f9b8ed57bd13"
server_key_iv: "d71269b190a73bbe9dde63d16e9a587b"
server_key_version: "encryptionkey@0"
I20260809 05:40:12.602636 28831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:40:12.606495 28850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:12.607982 28831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:12.608180 28831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal
uuid: "2813db21c200419aa7a061f5419bff26"
format_stamp: "Formatted at 2026-08-09 05:40:12 on dist-test-slave-3kk6"
server_key: "f463cff939cdbc4ff5d7f9b8ed57bd13"
server_key_iv: "d71269b190a73bbe9dde63d16e9a587b"
server_key_version: "encryptionkey@0"
I20260809 05:40:12.608424 28831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:12.632562 28831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:12.633512 28831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:12.633875 28831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:12.634967 28831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 05:40:12.636592 28831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 05:40:12.636725 28831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:12.636859 28831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 05:40:12.636956 28831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:12.670892 28650 heartbeater.cc:503] Master 127.30.137.254:45729 was elected leader, sending a full tablet report...
I20260809 05:40:12.673871 28831 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.195:46793
I20260809 05:40:12.673964 29005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.195:46793 every 8 connection(s)
I20260809 05:40:12.675352 28831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/data/info.pb
I20260809 05:40:12.685137 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 28831
I20260809 05:40:12.685348 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal/instance
I20260809 05:40:12.685915 31271 external_mini_cluster.cc:1428] Setting key de49e5d313e79665dffdd392c77d9739
I20260809 05:40:12.689144 29008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.678240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.195:36857 (local address 127.30.137.254:45729)
0809 05:40:12.678737 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.678741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.680001 (+ 1260us) server_negotiation.cc:407] Connection header received
0809 05:40:12.681782 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.681786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:12.681871 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:12.681970 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:12.683083 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.683846 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.685788 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.686248 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.686351 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.688153 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:12.688163 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:12.688169 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:12.688193 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:12.688248 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:12.688265 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:12.688362 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:12.688596 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 05:40:12.688695 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":248,"threads_started":1}
I20260809 05:40:12.690021 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.678943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:45729 (local address 127.30.137.195:36857)
0809 05:40:12.679846 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:12.679890 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:12.681531 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:12.682170 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:12.682178 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:12.682267 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:12.682911 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.682931 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.683975 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.683979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.685646 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.685661 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.686479 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.686485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.686866 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.687709 (+ 843us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 05:40:12.687715 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:12.687791 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 05:40:12.687925 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:12.688370 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:12.688379 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:12.688385 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 05:40:12.688530 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 05:40:12.688863 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":263,"threads_started":1}
I20260809 05:40:12.691413 29006 heartbeater.cc:347] Connected to a master server at 127.30.137.254:45729
I20260809 05:40:12.691748 29006 heartbeater.cc:464] Registering TS with master...
I20260809 05:40:12.692570 29006 heartbeater.cc:511] Master 127.30.137.254:45729 requested a full tablet report, sending...
I20260809 05:40:12.693996 28397 ts_manager.cc:194] Registered new tserver with Master: 2813db21c200419aa7a061f5419bff26 (127.30.137.195:46793)
I20260809 05:40:12.694911 28397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.137.195:36857
I20260809 05:40:12.701414 31271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 05:40:12.711315 29008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.705518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.30.137.254:45729)
0809 05:40:12.705694 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.705698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.706260 (+ 562us) server_negotiation.cc:407] Connection header received
0809 05:40:12.706416 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.706420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:12.706482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:12.706569 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:12.707105 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.707854 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.709172 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.709561 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.709654 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.710644 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:12.710704 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:12.710709 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:12.710734 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:12.710768 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:12.710781 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:12.710865 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:12.710987 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 05:40:12.711065 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:40:12.715361 28397 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 05:40:12.722268 28397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 05:40:12.757439 29026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.735552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:40177 (local address 127.0.0.1:40660)
0809 05:40:12.749529 (+ 13977us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:12.749549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:12.749785 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:12.750496 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:12.750501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:12.750526 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:12.750903 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.750913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.752694 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.752698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.754371 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.754385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.756354 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.756360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.756564 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.756619 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 05:40:12.756765 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 05:40:12.756981 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13843,"mutex_wait_us":29,"thread_start_us":12726,"threads_started":1}
I20260809 05:40:12.757501 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.735857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.30.137.194:40177)
0809 05:40:12.745263 (+ 9406us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.745278 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.749659 (+ 4381us) server_negotiation.cc:407] Connection header received
0809 05:40:12.750080 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.750089 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:12.750310 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:12.750503 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:12.751133 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.752540 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.754606 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.756153 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.756355 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.756592 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:12.756924 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 05:40:12.757136 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9136,"thread_start_us":9455,"threads_started":1}
I20260809 05:40:12.758181 29024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.735218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35040)
0809 05:40:12.746416 (+ 11198us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:12.746482 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:12.746795 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:12.748778 (+ 1983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:12.748785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:12.748831 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:12.749408 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.749424 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.751569 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.751575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.755033 (+ 3458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.755049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.757350 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.757358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.757580 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.757611 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:40:12.757697 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 05:40:12.757827 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10779,"mutex_wait_us":182,"thread_start_us":13970,"threads_started":1}
I20260809 05:40:12.758283 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.745483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.30.137.195:46793)
0809 05:40:12.746992 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.747000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.749004 (+ 2004us) server_negotiation.cc:407] Connection header received
0809 05:40:12.749147 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.749156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:12.749393 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:12.749619 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:12.750281 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.751876 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.754894 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.756361 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.756567 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.756779 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:12.757697 (+ 918us) server_negotiation.cc:299] Negotiation successful
0809 05:40:12.757875 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1337,"thread_start_us":1153,"threads_started":1}
I20260809 05:40:12.757545 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.735257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:46793 (local address 127.0.0.1:41874)
0809 05:40:12.748595 (+ 13338us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:12.748619 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:12.748800 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:12.749605 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:12.749609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:12.749636 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:12.750072 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.750086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.752063 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.752068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.754592 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:12.754610 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.756752 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.756760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:12.756916 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.756947 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 05:40:12.757033 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 05:40:12.757155 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13205,"mutex_wait_us":9,"thread_start_us":11646,"threads_started":1}
I20260809 05:40:12.759123 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.735753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.30.137.193:44731)
0809 05:40:12.747616 (+ 11863us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.747629 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.747675 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:40:12.747815 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.747826 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:12.748178 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:12.748423 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:12.749660 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.751397 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.755252 (+ 3855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:12.756748 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:12.756947 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:12.757190 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:12.758572 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0809 05:40:12.758776 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11622,"thread_start_us":12059,"threads_started":1}
I20260809 05:40:12.762226 28564 tablet_service.cc:1467] Processing CreateTablet for tablet e4717ba72a9b494c86c79ce1afdfaca2 (DEFAULT_TABLE table=test-table [id=c214662b51df4c269d52632b03e59b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:12.762235 28913 tablet_service.cc:1467] Processing CreateTablet for tablet e4717ba72a9b494c86c79ce1afdfaca2 (DEFAULT_TABLE table=test-table [id=c214662b51df4c269d52632b03e59b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:12.762235 28738 tablet_service.cc:1467] Processing CreateTablet for tablet e4717ba72a9b494c86c79ce1afdfaca2 (DEFAULT_TABLE table=test-table [id=c214662b51df4c269d52632b03e59b75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 05:40:12.763700 28564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4717ba72a9b494c86c79ce1afdfaca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:12.763753 28913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4717ba72a9b494c86c79ce1afdfaca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:12.763753 28738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4717ba72a9b494c86c79ce1afdfaca2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:12.776006 29035 tablet_bootstrap.cc:492] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: Bootstrap starting.
I20260809 05:40:12.780138 29035 tablet_bootstrap.cc:654] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:12.781234 29035 log.cc:824] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:12.783974 29038 tablet_bootstrap.cc:492] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Bootstrap starting.
I20260809 05:40:12.784798 29035 tablet_bootstrap.cc:492] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: No bootstrap required, opened a new log
I20260809 05:40:12.785084 29035 ts_tablet_manager.cc:1397] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 05:40:12.787187 29038 tablet_bootstrap.cc:654] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:12.788412 29038 log.cc:824] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:12.789124 29036 tablet_bootstrap.cc:492] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Bootstrap starting.
I20260809 05:40:12.792217 29036 tablet_bootstrap.cc:654] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:12.793409 29036 log.cc:824] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:12.793542 29035 raft_consensus.cc:348] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.794086 29035 raft_consensus.cc:374] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:12.794207 29035 raft_consensus.cc:729] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de64a53803ef46cb92d917bfce2c8db9, State: Initialized, Role: FOLLOWER
I20260809 05:40:12.794839 29038 tablet_bootstrap.cc:492] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: No bootstrap required, opened a new log
I20260809 05:40:12.795140 29038 ts_tablet_manager.cc:1397] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 05:40:12.795001 29035 consensus_queue.cc:260] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.796883 29035 ts_tablet_manager.cc:1428] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260809 05:40:12.797237 29036 tablet_bootstrap.cc:492] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: No bootstrap required, opened a new log
I20260809 05:40:12.797523 29036 ts_tablet_manager.cc:1397] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260809 05:40:12.804634 29038 raft_consensus.cc:348] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.805023 29038 raft_consensus.cc:374] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:12.805145 29038 raft_consensus.cc:729] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a98e3aabf0444231a36fc3ef16263577, State: Initialized, Role: FOLLOWER
I20260809 05:40:12.805706 29038 consensus_queue.cc:260] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.806860 28827 heartbeater.cc:503] Master 127.30.137.254:45729 was elected leader, sending a full tablet report...
I20260809 05:40:12.807523 29038 ts_tablet_manager.cc:1428] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 05:40:12.808425 29036 raft_consensus.cc:348] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.808795 29036 raft_consensus.cc:374] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:12.808899 29036 raft_consensus.cc:729] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2813db21c200419aa7a061f5419bff26, State: Initialized, Role: FOLLOWER
I20260809 05:40:12.809485 29036 consensus_queue.cc:260] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.810621 29006 heartbeater.cc:503] Master 127.30.137.254:45729 was elected leader, sending a full tablet report...
I20260809 05:40:12.811342 29036 ts_tablet_manager.cc:1428] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260809 05:40:12.895095 28651 tablet.cc:2369] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:40:12.913004 28828 tablet.cc:2369] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 05:40:12.927508 29007 tablet.cc:2369] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:12.996045 29043 raft_consensus.cc:482] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 05:40:12.996397 29043 raft_consensus.cc:504] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:12.998239 29043 leader_election.cc:288] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2813db21c200419aa7a061f5419bff26 (127.30.137.195:46793), a98e3aabf0444231a36fc3ef16263577 (127.30.137.194:40177)
I20260809 05:40:13.005507 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.998901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:41345 (local address 127.30.137.194:40177)
0809 05:40:12.999078 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.999085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:12.999739 (+ 654us) server_negotiation.cc:407] Connection header received
0809 05:40:12.999991 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:12.999995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:13.000073 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:13.000170 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:13.000754 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.001473 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.003616 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.004568 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.004745 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.004817 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:13.005110 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 05:40:13.005236 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 05:40:13.005599 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.998781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.194:40177 (local address 127.30.137.193:41345)
0809 05:40:12.999625 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:12.999642 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:12.999830 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:13.000193 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:13.000197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:13.000222 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:13.000578 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:13.000587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.001604 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.001611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:13.003422 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:13.003436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.004750 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.004756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:13.004944 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.004969 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 05:40:13.005075 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 05:40:13.005209 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":70,"thread_start_us":492,"threads_started":1}
I20260809 05:40:13.005647 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.998621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.137.193:40871 (local address 127.30.137.195:46793)
0809 05:40:12.998895 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:12.998901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:13.000116 (+ 1215us) server_negotiation.cc:407] Connection header received
0809 05:40:13.000320 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:13.000323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:13.000388 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:13.000485 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 05:40:13.001089 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.001799 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.003830 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.004766 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.004941 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.005009 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:13.005277 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 05:40:13.005410 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 05:40:13.006184 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:12.999263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.195:46793 (local address 127.30.137.193:40871)
0809 05:40:13.000011 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.000025 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:13.000177 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:13.000519 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:13.000523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:13.000549 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 05:40:13.000925 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:13.000934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.001940 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.001945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:13.003627 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:13.003640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.004968 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.004975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:13.005123 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.005146 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 05:40:13.005248 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 05:40:13.005374 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":431,"threads_started":1}
I20260809 05:40:13.006765 28763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4717ba72a9b494c86c79ce1afdfaca2" candidate_uuid: "de64a53803ef46cb92d917bfce2c8db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a98e3aabf0444231a36fc3ef16263577" is_pre_election: true
I20260809 05:40:13.007520 28763 raft_consensus.cc:2454] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de64a53803ef46cb92d917bfce2c8db9 in term 0.
I20260809 05:40:13.007350 28940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4717ba72a9b494c86c79ce1afdfaca2" candidate_uuid: "de64a53803ef46cb92d917bfce2c8db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2813db21c200419aa7a061f5419bff26" is_pre_election: true
I20260809 05:40:13.007953 28940 raft_consensus.cc:2454] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de64a53803ef46cb92d917bfce2c8db9 in term 0.
I20260809 05:40:13.008317 28500 leader_election.cc:302] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a98e3aabf0444231a36fc3ef16263577, de64a53803ef46cb92d917bfce2c8db9; no voters:
I20260809 05:40:13.008767 29043 raft_consensus.cc:2781] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 05:40:13.008965 29043 raft_consensus.cc:482] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 05:40:13.009080 29043 raft_consensus.cc:3037] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:13.011458 29043 raft_consensus.cc:504] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:13.012116 29043 leader_election.cc:288] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [CANDIDATE]: Term 1 election: Requested vote from peers 2813db21c200419aa7a061f5419bff26 (127.30.137.195:46793), a98e3aabf0444231a36fc3ef16263577 (127.30.137.194:40177)
I20260809 05:40:13.012601 28940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4717ba72a9b494c86c79ce1afdfaca2" candidate_uuid: "de64a53803ef46cb92d917bfce2c8db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2813db21c200419aa7a061f5419bff26"
I20260809 05:40:13.012629 28763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4717ba72a9b494c86c79ce1afdfaca2" candidate_uuid: "de64a53803ef46cb92d917bfce2c8db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a98e3aabf0444231a36fc3ef16263577"
I20260809 05:40:13.012799 28940 raft_consensus.cc:3037] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:13.012837 28763 raft_consensus.cc:3037] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:13.015209 28940 raft_consensus.cc:2454] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de64a53803ef46cb92d917bfce2c8db9 in term 1.
I20260809 05:40:13.015228 28763 raft_consensus.cc:2454] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de64a53803ef46cb92d917bfce2c8db9 in term 1.
I20260809 05:40:13.015667 28498 leader_election.cc:302] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2813db21c200419aa7a061f5419bff26, de64a53803ef46cb92d917bfce2c8db9; no voters:
I20260809 05:40:13.015974 29043 raft_consensus.cc:2781] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:13.016456 29043 raft_consensus.cc:686] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 1 LEADER]: Becoming Leader. State: Replica: de64a53803ef46cb92d917bfce2c8db9, State: Running, Role: LEADER
I20260809 05:40:13.016942 29043 consensus_queue.cc:237] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } }
I20260809 05:40:13.021998 28397 catalog_manager.cc:5495] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 reported cstate change: term changed from 0 to 1, leader changed from <none> to de64a53803ef46cb92d917bfce2c8db9 (127.30.137.193). New cstate: current_term: 1 leader_uuid: "de64a53803ef46cb92d917bfce2c8db9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de64a53803ef46cb92d917bfce2c8db9" member_type: VOTER last_known_addr { host: "127.30.137.193" port: 44731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 } health_report { overall_health: UNKNOWN } } }
I20260809 05:40:13.056962 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.051523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.30.137.193:44731)
0809 05:40:13.051751 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:13.051759 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:13.051785 (+ 26us) server_negotiation.cc:407] Connection header received
0809 05:40:13.051900 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:13.051905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:13.052013 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:13.052175 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:40:13.052564 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.053270 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.054720 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.055159 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.055301 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.055793 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:40:13.056302 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:40:13.056508 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:13.056625 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 05:40:13.056709 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:40:13.066131 28940 raft_consensus.cc:1261] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 1 FOLLOWER]: Refusing update from remote peer de64a53803ef46cb92d917bfce2c8db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:13.066131 28763 raft_consensus.cc:1261] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 1 FOLLOWER]: Refusing update from remote peer de64a53803ef46cb92d917bfce2c8db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:13.067204 29050 consensus_queue.cc:1035] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a98e3aabf0444231a36fc3ef16263577" member_type: VOTER last_known_addr { host: "127.30.137.194" port: 40177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:13.067627 29043 consensus_queue.cc:1035] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2813db21c200419aa7a061f5419bff26" member_type: VOTER last_known_addr { host: "127.30.137.195" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:13.083107 29056 mvcc.cc:205] Tried to move back new op lower bound from 7316496437507215360 to 7316496437321412608. Current Snapshot: MvccSnapshot[applied={T|T < 7316496437507215360}]
I20260809 05:40:13.098590 28397 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:13.098946 28397 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 05:40:13.102769 28397 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 896c88eeb7844b958ee570dbfc18ce5b: Sending DeleteTablet for 3 replicas of tablet e4717ba72a9b494c86c79ce1afdfaca2
I20260809 05:40:13.104110 28913 tablet_service.cc:1514] Processing DeleteTablet for tablet e4717ba72a9b494c86c79ce1afdfaca2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:13 UTC) from {username='slave'} at 127.0.0.1:41874
I20260809 05:40:13.104488 28738 tablet_service.cc:1514] Processing DeleteTablet for tablet e4717ba72a9b494c86c79ce1afdfaca2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:13 UTC) from {username='slave'} at 127.0.0.1:40660
I20260809 05:40:13.104977 28564 tablet_service.cc:1514] Processing DeleteTablet for tablet e4717ba72a9b494c86c79ce1afdfaca2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 05:40:13 UTC) from {username='slave'} at 127.0.0.1:35040
I20260809 05:40:13.106603 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 28474
I20260809 05:40:13.112783 29068 tablet_replica.cc:330] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: stopping tablet replica
I20260809 05:40:13.113322 29068 raft_consensus.cc:2229] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:13.113404 29073 tablet_replica.cc:330] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9: stopping tablet replica
I20260809 05:40:13.113953 29068 raft_consensus.cc:2258] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:13.115080 29071 tablet_replica.cc:330] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: stopping tablet replica
I20260809 05:40:13.115489 29071 raft_consensus.cc:2229] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 05:40:13.116010 29071 raft_consensus.cc:2258] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:13.117167 29068 ts_tablet_manager.cc:1905] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:13.118113 29071 ts_tablet_manager.cc:1905] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 05:40:13.121424 29028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.107966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.30.137.193:44731)
0809 05:40:13.108198 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:13.108219 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:13.112133 (+ 3914us) server_negotiation.cc:407] Connection header received
0809 05:40:13.112357 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:13.112366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:13.112517 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:13.112645 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:13.113829 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.115159 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.117358 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.118031 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.118136 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.119684 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:13.119697 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:13.119713 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:13.120490 (+ 777us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:13.120559 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:13.120583 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:13.120658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:13.120921 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 05:40:13.121031 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 05:40:13.122714 29073 raft_consensus.cc:2229] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 05:40:13.122802 28645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:13.125743 29068 ts_tablet_manager.cc:1918] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:13.125949 29068 log.cc:1196] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-2/wal/wals/e4717ba72a9b494c86c79ce1afdfaca2
I20260809 05:40:13.126411 29068 ts_tablet_manager.cc:1939] T e4717ba72a9b494c86c79ce1afdfaca2 P 2813db21c200419aa7a061f5419bff26: Deleting consensus metadata
I20260809 05:40:13.126981 29071 ts_tablet_manager.cc:1918] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 05:40:13.127146 29071 log.cc:1196] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Deleting WAL directory at /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786253675866932-31271-0/minicluster-data/ts-1/wal/wals/e4717ba72a9b494c86c79ce1afdfaca2
I20260809 05:40:13.127678 29071 ts_tablet_manager.cc:1939] T e4717ba72a9b494c86c79ce1afdfaca2 P a98e3aabf0444231a36fc3ef16263577: Deleting consensus metadata
I20260809 05:40:13.128341 28376 catalog_manager.cc:4841] TS 2813db21c200419aa7a061f5419bff26 (127.30.137.195:46793): tablet e4717ba72a9b494c86c79ce1afdfaca2 (table test-table [id=c214662b51df4c269d52632b03e59b75]) successfully deleted
I20260809 05:40:13.129448 28379 catalog_manager.cc:4841] TS a98e3aabf0444231a36fc3ef16263577 (127.30.137.194:40177): tablet e4717ba72a9b494c86c79ce1afdfaca2 (table test-table [id=c214662b51df4c269d52632b03e59b75]) successfully deleted
I20260809 05:40:13.220142 29073 raft_consensus.cc:2258] T e4717ba72a9b494c86c79ce1afdfaca2 P de64a53803ef46cb92d917bfce2c8db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 05:40:13.220665 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 28474
W20260809 05:40:13.245607 28380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.137.193:44731 (error 108)
I20260809 05:40:13.246954 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 28654
I20260809 05:40:13.247454 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.246927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35054)
0809 05:40:13.247139 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.247274 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 05:40:13.247673 28380 catalog_manager.cc:4584] TS de64a53803ef46cb92d917bfce2c8db9 (127.30.137.193:44731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4717ba72a9b494c86c79ce1afdfaca2: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
I20260809 05:40:13.254315 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.248363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.30.137.194:40177)
0809 05:40:13.248520 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:13.248524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:13.248540 (+ 16us) server_negotiation.cc:407] Connection header received
0809 05:40:13.248599 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:13.248603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:13.248699 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:13.248785 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:13.249161 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.250086 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.251390 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.251795 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.251891 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.252895 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:13.252904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:13.252917 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:13.253563 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:13.253609 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:13.253635 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:13.253724 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:13.253860 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 05:40:13.253950 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 05:40:13.255153 28816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:13.301529 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.300954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35070)
0809 05:40:13.301241 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.301365 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260809 05:40:13.302609 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.302286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35086)
0809 05:40:13.302418 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.302503 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 05:40:13.347756 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 28654
I20260809 05:40:13.363153 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.362602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35088)
0809 05:40:13.362860 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.362961 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 05:40:13.364305 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.363982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35098)
0809 05:40:13.364122 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.364194 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 05:40:13.376547 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 28831
I20260809 05:40:13.383445 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.377365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.30.137.195:46793)
0809 05:40:13.377556 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:13.377561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:13.377581 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:40:13.377691 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:13.377695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:13.377792 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:13.377891 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:13.378279 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.379071 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.380386 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:13.380804 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:13.380912 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:13.381955 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:13.381964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:13.381978 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:13.382548 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:13.382599 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:13.382621 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:13.382878 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:13.382986 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 05:40:13.383076 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 05:40:13.384143 28999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:13.445251 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.444668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35104)
0809 05:40:13.444939 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.445053 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260809 05:40:13.446429 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.446012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35108)
0809 05:40:13.446232 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.446319 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 05:40:13.474687 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 28831
I20260809 05:40:13.502241 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 28358
I20260809 05:40:13.503538 28438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:13.620004 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:13.619311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.193:44731 (local address 127.0.0.1:35124)
0809 05:40:13.619556 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:13.619706 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.193:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 05:40:13.620354 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 28358
2026-08-09T05:40:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3156 ms)
[----------] 6 tests from EncryptionPolicyTest (24937 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:13 dist-test-slave-3kk6 krb5kdc[29086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:13 dist-test-slave-3kk6 krb5kdc[29086](info): set up 2 sockets
Aug 09 05:40:13 dist-test-slave-3kk6 krb5kdc[29086](info): commencing operation
krb5kdc: starting...
W20260809 05:40:14.986020 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:15 dist-test-slave-3kk6 krb5kdc[29086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:15.344573 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:15.704234 29104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:15.704605 29104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:15.704771 29104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:15.712208 29104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:15.712379 29104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:15.712484 29104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:15.712592 29104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:15.712671 29104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:15.722640 29104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:15.724092 29104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:15.725419 29104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:15.737532 29117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:15.737812 29114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:15.737775 29115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:15.738952 29104 server_base.cc:1034] running on GCE node
I20260809 05:40:15.739728 29104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:15.741137 29104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:15.742344 29104 hybrid_clock.cc:648] HybridClock initialized: now 1786254015742286 us; error 53 us; skew 500 ppm
I20260809 05:40:15.742782 29104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:15 dist-test-slave-3kk6 krb5kdc[29086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:15.746117 29104 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:15.747679 29104 webserver.cc:460] Webserver started at http://127.0.0.1:42327/ using document root <none> and password file <none>
I20260809 05:40:15.748450 29104 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:15.748621 29104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:15.748968 29104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:15.752300 29104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "97a701246cab4b17bb2d51de3cfef052"
format_stamp: "Formatted at 2026-08-09 05:40:15 on dist-test-slave-3kk6"
server_key: "bf0380d3c39c2c42c8c7fe929d4cd1b8"
server_key_iv: "c4b61476279b4a28346c82fd2be7bf81"
server_key_version: "encryptionkey@0"
I20260809 05:40:15.753140 29104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "97a701246cab4b17bb2d51de3cfef052"
format_stamp: "Formatted at 2026-08-09 05:40:15 on dist-test-slave-3kk6"
server_key: "bf0380d3c39c2c42c8c7fe929d4cd1b8"
server_key_iv: "c4b61476279b4a28346c82fd2be7bf81"
server_key_version: "encryptionkey@0"
I20260809 05:40:15.758684 29104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:40:15.762197 29128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:15.763595 29104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:15.763813 29104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "97a701246cab4b17bb2d51de3cfef052"
format_stamp: "Formatted at 2026-08-09 05:40:15 on dist-test-slave-3kk6"
server_key: "bf0380d3c39c2c42c8c7fe929d4cd1b8"
server_key_iv: "c4b61476279b4a28346c82fd2be7bf81"
server_key_version: "encryptionkey@0"
I20260809 05:40:15.764030 29104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:15.784354 29104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:15.788743 29104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:15.789093 29104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:15.808902 29104 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37759
I20260809 05:40:15.808936 29206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37759 every 8 connection(s)
I20260809 05:40:15.810392 29104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:15.814287 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:15.820362 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 29104
I20260809 05:40:15.820618 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:15.821527 31271 external_mini_cluster.cc:1428] Setting key 9529aaf9e9b60668e2edd4b8b766fb92
I20260809 05:40:15.825987 29207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052: Bootstrap starting.
I20260809 05:40:15.829882 29207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:15.831117 29207 log.cc:824] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:15.834275 29207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052: No bootstrap required, opened a new log
Aug 09 05:40:15 dist-test-slave-3kk6 krb5kdc[29086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:40:15.842931 29207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a701246cab4b17bb2d51de3cfef052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260809 05:40:15.843370 29207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:15.843497 29207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a701246cab4b17bb2d51de3cfef052, State: Initialized, Role: FOLLOWER
I20260809 05:40:15.844251 29207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a701246cab4b17bb2d51de3cfef052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260809 05:40:15.844575 29207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:15.844735 29207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:15.844946 29207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:15.846997 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:15.823731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52637 (local address 127.0.0.1:37759)
0809 05:40:15.824581 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:15.824593 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:15.824635 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:40:15.826476 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:15.826502 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:15.826952 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:15.827268 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:15.829241 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:15.830716 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:15.832402 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:15.833058 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:15.833194 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:15.838387 (+ 5193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:15.838397 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:15.838412 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:15.838449 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:15.841933 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:15.842857 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:15.842864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:15.842870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:15.843045 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:15.843669 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:15.843674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:15.843677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:15.844449 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:15.844805 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:15.845467 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 05:40:15.845850 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":427,"threads_started":1}
I20260809 05:40:15.847723 29207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a701246cab4b17bb2d51de3cfef052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260809 05:40:15.848266 29207 leader_election.cc:302] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97a701246cab4b17bb2d51de3cfef052; no voters:
I20260809 05:40:15.849030 29207 leader_election.cc:288] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:15.849160 29212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:15.850917 29212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [term 1 LEADER]: Becoming Leader. State: Replica: 97a701246cab4b17bb2d51de3cfef052, State: Running, Role: LEADER
I20260809 05:40:15.851531 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:15.851521 29212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a701246cab4b17bb2d51de3cfef052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } }
I20260809 05:40:15.852035 29207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:15.854589 29214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a701246cab4b17bb2d51de3cfef052. Latest consensus state: current_term: 1 leader_uuid: "97a701246cab4b17bb2d51de3cfef052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a701246cab4b17bb2d51de3cfef052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } } }
I20260809 05:40:15.855050 29214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:15.856833 29213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97a701246cab4b17bb2d51de3cfef052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a701246cab4b17bb2d51de3cfef052" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37759 } } }
I20260809 05:40:15.857317 29213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:15.859649 29219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:15.866250 29219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:15.887760 29219 catalog_manager.cc:1347] Generated new cluster ID: e62ee775e7d34b74b0f4cb0d7cfcb517
I20260809 05:40:15.887895 29219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:15.890062 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:15.863253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36957 (local address 127.0.0.1:37759)
0809 05:40:15.866630 (+ 3377us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:15.866636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:15.869232 (+ 2596us) server_negotiation.cc:407] Connection header received
0809 05:40:15.869538 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:15.869549 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:15.869648 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:15.869785 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:15.872163 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:15.873778 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:15.876303 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:15.877285 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:15.877458 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:15.882107 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:15.882118 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:15.882126 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:15.882187 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:15.886307 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:15.887497 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:15.887505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:15.887508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:15.887658 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:15.888288 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:15.888294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:15.888297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:15.888686 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:15.889011 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:15.889467 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 05:40:15.889659 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3237}
I20260809 05:40:15.913586 29219 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:15.915067 29219 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:15.928192 29219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97a701246cab4b17bb2d51de3cfef052: Generated new TSK 0
I20260809 05:40:15.929260 29219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:15.944639 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29104
I20260809 05:40:15.946480 29199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:16.040799 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 29104
W20260809 05:40:16.068284 29083 connection.cc:535] client connection to 127.0.0.1:37759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37759): Connection reset by peer (error 104)
2026-08-09T05:40:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:16 dist-test-slave-3kk6 krb5kdc[29255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:16 dist-test-slave-3kk6 krb5kdc[29255](info): set up 2 sockets
Aug 09 05:40:16 dist-test-slave-3kk6 krb5kdc[29255](info): commencing operation
krb5kdc: starting...
W20260809 05:40:17.383893 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:17 dist-test-slave-3kk6 krb5kdc[29255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:17.739424 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:18.104346 29275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:18.104900 29275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:18.105096 29275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:18.112991 29275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:18.113127 29275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:18.113238 29275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:18.113346 29275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:18.113430 29275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:18.123823 29275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43649
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:18.125176 29275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:18.126546 29275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:18.139264 29284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:18.139357 29285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:18.139357 29288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:18.140450 29275 server_base.cc:1034] running on GCE node
I20260809 05:40:18.141327 29275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:18.142853 29275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:18.144047 29275 hybrid_clock.cc:648] HybridClock initialized: now 1786254018144010 us; error 31 us; skew 500 ppm
I20260809 05:40:18.144429 29275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:18 dist-test-slave-3kk6 krb5kdc[29255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:18.148041 29275 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:18.149713 29275 webserver.cc:460] Webserver started at http://127.0.0.1:44967/ using document root <none> and password file <none>
I20260809 05:40:18.150430 29275 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:18.150570 29275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:18.150951 29275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:18.154311 29275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "a17af62b68334d639e5a90bf4fcc5a31"
format_stamp: "Formatted at 2026-08-09 05:40:18 on dist-test-slave-3kk6"
server_key: "fc8ad66ec78d38a8aad4962305bf3291"
server_key_iv: "3c477ce36d63dd536a0ad24a5104d88a"
server_key_version: "encryptionkey@0"
I20260809 05:40:18.155198 29275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "a17af62b68334d639e5a90bf4fcc5a31"
format_stamp: "Formatted at 2026-08-09 05:40:18 on dist-test-slave-3kk6"
server_key: "fc8ad66ec78d38a8aad4962305bf3291"
server_key_iv: "3c477ce36d63dd536a0ad24a5104d88a"
server_key_version: "encryptionkey@0"
I20260809 05:40:18.160633 29275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:18.164429 29299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:18.165931 29275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:40:18.166152 29275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "a17af62b68334d639e5a90bf4fcc5a31"
format_stamp: "Formatted at 2026-08-09 05:40:18 on dist-test-slave-3kk6"
server_key: "fc8ad66ec78d38a8aad4962305bf3291"
server_key_iv: "3c477ce36d63dd536a0ad24a5104d88a"
server_key_version: "encryptionkey@0"
I20260809 05:40:18.166385 29275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:18.179311 29275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:18.184000 29275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:18.184317 29275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:18.204201 29275 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43649
I20260809 05:40:18.204233 29371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43649 every 8 connection(s)
I20260809 05:40:18.205724 29275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:18.209575 29372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:18.215557 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 29275
I20260809 05:40:18.215766 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:18.216225 31271 external_mini_cluster.cc:1428] Setting key d6a0fc44eda7128280febc092f9518bb
I20260809 05:40:18.220789 29372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31: Bootstrap starting.
I20260809 05:40:18.224421 29372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:18.225713 29372 log.cc:824] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:18.229094 29372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31: No bootstrap required, opened a new log
I20260809 05:40:18.230826 29379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:18.218107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45105 (local address 127.0.0.1:43649)
0809 05:40:18.219073 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:18.219086 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:18.219131 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:40:18.220959 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:18.220992 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:18.221396 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:18.221712 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:18.222381 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:18.223791 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:18.225801 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:18.226926 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:18.227141 (+ 215us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:18.228442 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:18.228449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:18.228462 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:18.229003 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:18.229089 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:18.229113 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:18.229283 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:18.229609 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 05:40:18.229796 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":414,"threads_started":1}
I20260809 05:40:18.234912 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:18.237830 29372 raft_consensus.cc:348] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17af62b68334d639e5a90bf4fcc5a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43649 } }
I20260809 05:40:18.238267 29372 raft_consensus.cc:374] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:18.238394 29372 raft_consensus.cc:729] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17af62b68334d639e5a90bf4fcc5a31, State: Initialized, Role: FOLLOWER
I20260809 05:40:18.239063 29372 consensus_queue.cc:260] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17af62b68334d639e5a90bf4fcc5a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43649 } }
I20260809 05:40:18.239409 29372 raft_consensus.cc:388] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:18.239593 29372 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:18.239840 29372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:18.242331 29372 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17af62b68334d639e5a90bf4fcc5a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43649 } }
I20260809 05:40:18.242964 29372 leader_election.cc:302] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a17af62b68334d639e5a90bf4fcc5a31; no voters:
I20260809 05:40:18.243693 29372 leader_election.cc:288] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:18.243860 29388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:18.246577 29372 sys_catalog.cc:564] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:18.247131 29379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:18.239649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50431 (local address 127.0.0.1:43649)
0809 05:40:18.239817 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:18.239822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:18.240371 (+ 549us) server_negotiation.cc:407] Connection header received
0809 05:40:18.240614 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:18.240618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:18.240683 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:18.240810 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:18.241313 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:18.242102 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:18.243693 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:18.244635 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:18.244753 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:18.246254 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:18.246259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:18.246263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:18.246281 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:18.246346 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:18.246362 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:18.246488 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:18.246730 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 05:40:18.246786 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:40:18.251031 29388 raft_consensus.cc:686] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [term 1 LEADER]: Becoming Leader. State: Replica: a17af62b68334d639e5a90bf4fcc5a31, State: Running, Role: LEADER
I20260809 05:40:18.251791 29388 consensus_queue.cc:237] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17af62b68334d639e5a90bf4fcc5a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43649 } }
I20260809 05:40:18.255143 29389 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a17af62b68334d639e5a90bf4fcc5a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17af62b68334d639e5a90bf4fcc5a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43649 } } }
I20260809 05:40:18.255566 29389 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:18.255388 29391 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a17af62b68334d639e5a90bf4fcc5a31. Latest consensus state: current_term: 1 leader_uuid: "a17af62b68334d639e5a90bf4fcc5a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17af62b68334d639e5a90bf4fcc5a31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43649 } } }
I20260809 05:40:18.255740 29391 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31 [sys.catalog]: This master's current role is: LEADER
W20260809 05:40:18.261107 29411 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:40:18.261252 29411 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:40:18.261422 29412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:18.263381 29412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:18.273016 29412 catalog_manager.cc:1347] Generated new cluster ID: 7a6053c29b2c48e7a4bfcc1f8a77145e
I20260809 05:40:18.273166 29412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:18.284165 29412 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:18.285892 29412 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:18.301591 29412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a17af62b68334d639e5a90bf4fcc5a31: Generated new TSK 0
I20260809 05:40:18.302479 29412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:18.312609 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29275
W20260809 05:40:18.313889 29364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45105
W20260809 05:40:18.314913 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29275: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:18.315119 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 29275
W20260809 05:40:18.341095 29253 connection.cc:535] client connection to 127.0.0.1:43649 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43649): Connection reset by peer (error 104)
2026-08-09T05:40:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2272 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:18 dist-test-slave-3kk6 krb5kdc[29422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:18 dist-test-slave-3kk6 krb5kdc[29422](info): set up 2 sockets
Aug 09 05:40:18 dist-test-slave-3kk6 krb5kdc[29422](info): commencing operation
krb5kdc: starting...
W20260809 05:40:19.679070 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:19 dist-test-slave-3kk6 krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:20.041661 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:20.410058 29442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:20.410399 29442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:20.410567 29442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:20.417997 29442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:20.418131 29442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:20.418188 29442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:20.418294 29442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:20.418367 29442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:20.428745 29442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39575
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:20.430212 29442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:20.431628 29442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:20.444479 29453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:20.444396 29450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:20.444784 29449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:20.445676 29442 server_base.cc:1034] running on GCE node
I20260809 05:40:20.446600 29442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:20.448275 29442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:20.449540 29442 hybrid_clock.cc:648] HybridClock initialized: now 1786254020449492 us; error 50 us; skew 500 ppm
I20260809 05:40:20.449968 29442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:20 dist-test-slave-3kk6 krb5kdc[29422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254020, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:20.458055 29442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:20.459887 29442 webserver.cc:460] Webserver started at http://127.0.0.1:42305/ using document root <none> and password file <none>
I20260809 05:40:20.460680 29442 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:20.460824 29442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:20.461164 29442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:20.464488 29442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "6736d63a2b9144639564df6f2069c282"
format_stamp: "Formatted at 2026-08-09 05:40:20 on dist-test-slave-3kk6"
server_key: "a067f29b65e43dbd8363b8e55e17446b"
server_key_iv: "06502ddca5a1faf50d455314d2b0c85e"
server_key_version: "encryptionkey@0"
I20260809 05:40:20.465332 29442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "6736d63a2b9144639564df6f2069c282"
format_stamp: "Formatted at 2026-08-09 05:40:20 on dist-test-slave-3kk6"
server_key: "a067f29b65e43dbd8363b8e55e17446b"
server_key_iv: "06502ddca5a1faf50d455314d2b0c85e"
server_key_version: "encryptionkey@0"
I20260809 05:40:20.471165 29442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 05:40:20.474920 29462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:20.476419 29442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:40:20.476634 29442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "6736d63a2b9144639564df6f2069c282"
format_stamp: "Formatted at 2026-08-09 05:40:20 on dist-test-slave-3kk6"
server_key: "a067f29b65e43dbd8363b8e55e17446b"
server_key_iv: "06502ddca5a1faf50d455314d2b0c85e"
server_key_version: "encryptionkey@0"
I20260809 05:40:20.476912 29442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:20.483300 29442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:20.487927 29442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:20.488283 29442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:20.508101 29442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39575
I20260809 05:40:20.508149 29539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39575 every 8 connection(s)
I20260809 05:40:20.509652 29442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:20.513945 29541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:20.518931 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 29442
I20260809 05:40:20.519135 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:20.519584 31271 external_mini_cluster.cc:1428] Setting key 8a4dd8b14fce1797a94992cf743d6e41
I20260809 05:40:20.526993 29541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282: Bootstrap starting.
I20260809 05:40:20.529160 29546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:20.521612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44749 (local address 127.0.0.1:39575)
0809 05:40:20.522532 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:20.522545 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:20.522588 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:40:20.524932 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:20.524957 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:20.525311 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:20.525656 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:20.527005 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:20.527012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:20.527025 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:20.527041 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:20.527142 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:20.527184 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:20.527380 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:20.527512 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 05:40:20.528064 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":455,"threads_started":1}
I20260809 05:40:20.531064 29541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:20.532498 29541 log.cc:824] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:20.533231 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:20.535831 29541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282: No bootstrap required, opened a new log
I20260809 05:40:20.540608 29546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:20.537926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38805 (local address 127.0.0.1:39575)
0809 05:40:20.538152 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:20.538159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:20.538453 (+ 294us) server_negotiation.cc:407] Connection header received
0809 05:40:20.538615 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:20.538621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:20.538720 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:20.538870 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:20.539875 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:20.539881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:20.539888 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:20.539915 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:20.539968 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:20.539990 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:20.540129 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:20.540185 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 05:40:20.540261 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 05:40:20.546134 29541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6736d63a2b9144639564df6f2069c282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260809 05:40:20.546612 29541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:20.546839 29541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6736d63a2b9144639564df6f2069c282, State: Initialized, Role: FOLLOWER
I20260809 05:40:20.547638 29541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6736d63a2b9144639564df6f2069c282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260809 05:40:20.547993 29541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:20.548197 29541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:20.548415 29541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:20.551119 29541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6736d63a2b9144639564df6f2069c282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260809 05:40:20.551776 29541 leader_election.cc:302] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6736d63a2b9144639564df6f2069c282; no voters:
I20260809 05:40:20.552485 29541 leader_election.cc:288] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:20.552659 29557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:20.554153 29557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [term 1 LEADER]: Becoming Leader. State: Replica: 6736d63a2b9144639564df6f2069c282, State: Running, Role: LEADER
I20260809 05:40:20.555047 29557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6736d63a2b9144639564df6f2069c282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } }
I20260809 05:40:20.555588 29541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:20.558430 29561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6736d63a2b9144639564df6f2069c282. Latest consensus state: current_term: 1 leader_uuid: "6736d63a2b9144639564df6f2069c282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6736d63a2b9144639564df6f2069c282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } } }
I20260809 05:40:20.558993 29561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:20.559046 29558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6736d63a2b9144639564df6f2069c282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6736d63a2b9144639564df6f2069c282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39575 } } }
I20260809 05:40:20.559427 29558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:20.561221 29569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:20.567447 29569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:20.578238 29569 catalog_manager.cc:1347] Generated new cluster ID: 8f69dd0d62e9405488527662cd783c6c
I20260809 05:40:20.578447 29569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:20.591445 29569 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:20.592020 29569 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:20.600960 29569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6736d63a2b9144639564df6f2069c282: Generated new TSK 0
I20260809 05:40:20.602094 29569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:20.615952 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29442
W20260809 05:40:20.617285 29533 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44749
W20260809 05:40:20.618095 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29442: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:20.618271 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 29442
2026-08-09T05:40:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:20 dist-test-slave-3kk6 krb5kdc[29596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:20 dist-test-slave-3kk6 krb5kdc[29596](info): set up 2 sockets
Aug 09 05:40:20 dist-test-slave-3kk6 krb5kdc[29596](info): commencing operation
krb5kdc: starting...
W20260809 05:40:22.005353 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:22 dist-test-slave-3kk6 krb5kdc[29596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:22.386449 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37933
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:22.753629 29615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:22.753965 29615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:22.754156 29615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:22.761834 29615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:22.761997 29615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:22.762096 29615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:22.762209 29615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:22.762297 29615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:22.772838 29615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:22.774302 29615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:22.775731 29615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:22.789330 29623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:22.789400 29622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:22.789400 29626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:22.790802 29615 server_base.cc:1034] running on GCE node
I20260809 05:40:22.791756 29615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:22.793414 29615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:22.794714 29615 hybrid_clock.cc:648] HybridClock initialized: now 1786254022794623 us; error 49 us; skew 500 ppm
I20260809 05:40:22.795135 29615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:22 dist-test-slave-3kk6 krb5kdc[29596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254022, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:22.798544 29615 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:22.800216 29615 webserver.cc:460] Webserver started at http://127.0.0.1:32777/ using document root <none> and password file <none>
I20260809 05:40:22.800985 29615 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:22.801143 29615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:22.801496 29615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:22.804803 29615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "a2a5a98318aa4b52b77c1be93860dafa"
format_stamp: "Formatted at 2026-08-09 05:40:22 on dist-test-slave-3kk6"
server_key: "be55d650a86871ad32dc4e86b50ef35d"
server_key_iv: "49093a638d5ec5444914d9bfafc83cc8"
server_key_version: "encryptionkey@0"
I20260809 05:40:22.805661 29615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "a2a5a98318aa4b52b77c1be93860dafa"
format_stamp: "Formatted at 2026-08-09 05:40:22 on dist-test-slave-3kk6"
server_key: "be55d650a86871ad32dc4e86b50ef35d"
server_key_iv: "49093a638d5ec5444914d9bfafc83cc8"
server_key_version: "encryptionkey@0"
I20260809 05:40:22.811340 29615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:40:22.815034 29634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:22.816370 29615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:40:22.816572 29615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "a2a5a98318aa4b52b77c1be93860dafa"
format_stamp: "Formatted at 2026-08-09 05:40:22 on dist-test-slave-3kk6"
server_key: "be55d650a86871ad32dc4e86b50ef35d"
server_key_iv: "49093a638d5ec5444914d9bfafc83cc8"
server_key_version: "encryptionkey@0"
I20260809 05:40:22.816787 29615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:22.832636 29615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:22.837337 29615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:22.837692 29615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:22.858240 29615 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37933
I20260809 05:40:22.858274 29715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37933 every 8 connection(s)
I20260809 05:40:22.859903 29615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:22.863058 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 29615
I20260809 05:40:22.863274 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:22.863857 31271 external_mini_cluster.cc:1428] Setting key 947ffc7a82425b8718f664ac9f24d977
I20260809 05:40:22.864598 29716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:40:22 dist-test-slave-3kk6 krb5kdc[29596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:40:22.881052 29716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa: Bootstrap starting.
I20260809 05:40:22.885123 29716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:22.886554 29716 log.cc:824] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:22.889631 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:22.865819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54951 (local address 127.0.0.1:37933)
0809 05:40:22.866695 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:22.866711 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:22.866762 (+ 51us) server_negotiation.cc:407] Connection header received
0809 05:40:22.868359 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:22.868382 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:22.868815 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:22.869174 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:22.871276 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:22.872692 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:22.874554 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:22.875379 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:22.875537 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:22.881058 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:22.881067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:22.881084 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:22.881124 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:22.884689 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:22.885829 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:22.885836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:22.885843 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:22.886029 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:22.886525 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:22.886529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:22.886534 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:22.887373 (+ 839us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:22.887727 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:22.888140 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 05:40:22.888622 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":390,"threads_started":1}
I20260809 05:40:22.889991 29716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa: No bootstrap required, opened a new log
I20260809 05:40:22.893906 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:22.901153 29716 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a5a98318aa4b52b77c1be93860dafa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37933 } }
I20260809 05:40:22.901777 29716 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:22.901921 29716 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a5a98318aa4b52b77c1be93860dafa, State: Initialized, Role: FOLLOWER
I20260809 05:40:22.902727 29716 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a5a98318aa4b52b77c1be93860dafa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37933 } }
I20260809 05:40:22.903002 29716 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:22.903162 29716 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:22.903344 29716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:22.906414 29716 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a5a98318aa4b52b77c1be93860dafa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37933 } }
I20260809 05:40:22.907178 29716 leader_election.cc:302] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2a5a98318aa4b52b77c1be93860dafa; no voters:
I20260809 05:40:22.907980 29716 leader_election.cc:288] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:22.908123 29730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:22.910354 29716 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:22.910981 29730 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [term 1 LEADER]: Becoming Leader. State: Replica: a2a5a98318aa4b52b77c1be93860dafa, State: Running, Role: LEADER
I20260809 05:40:22.911841 29730 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a5a98318aa4b52b77c1be93860dafa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37933 } }
I20260809 05:40:22.920536 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:22.899870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49729 (local address 127.0.0.1:37933)
0809 05:40:22.900093 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:22.900105 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:22.901295 (+ 1190us) server_negotiation.cc:407] Connection header received
0809 05:40:22.901463 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:22.901468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:22.901557 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:22.901682 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:22.903668 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:22.904862 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:22.906142 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:22.906879 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:22.907057 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:22.910959 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:22.910972 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:22.910979 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:22.911035 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:22.915851 (+ 4816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:22.917230 (+ 1379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:22.917246 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:22.917253 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:22.917536 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:22.918116 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:22.918125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:22.918130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:22.918593 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:22.919018 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:22.919355 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 05:40:22.919625 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:40:22.919631 29731 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2a5a98318aa4b52b77c1be93860dafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a5a98318aa4b52b77c1be93860dafa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37933 } } }
I20260809 05:40:22.920976 29731 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:22.920033 29732 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2a5a98318aa4b52b77c1be93860dafa. Latest consensus state: current_term: 1 leader_uuid: "a2a5a98318aa4b52b77c1be93860dafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a5a98318aa4b52b77c1be93860dafa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37933 } } }
I20260809 05:40:22.921947 29732 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:22.928032 29750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:22.933799 29750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:22.943434 29750 catalog_manager.cc:1347] Generated new cluster ID: 6cf62e8a55c043d9ae6b075fab8ca37e
I20260809 05:40:22.943589 29750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:22.958465 29750 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:22.960624 29750 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:22.980211 29750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2a5a98318aa4b52b77c1be93860dafa: Generated new TSK 0
I20260809 05:40:22.981164 29750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:22.996577 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29615
I20260809 05:40:22.998507 29709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:23.096037 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 29615
2026-08-09T05:40:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:23 dist-test-slave-3kk6 krb5kdc[29759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:23 dist-test-slave-3kk6 krb5kdc[29759](info): set up 2 sockets
Aug 09 05:40:23 dist-test-slave-3kk6 krb5kdc[29759](info): commencing operation
krb5kdc: starting...
W20260809 05:40:24.450958 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.002s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:24 dist-test-slave-3kk6 krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:24.804209 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:25.160575 29777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:25.160998 29777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:25.161163 29777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:25.168429 29777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:25.168586 29777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:25.168708 29777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:25.168816 29777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:25.168902 29777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:25.178853 29777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:25.180167 29777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:25.181550 29777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:25.193670 29789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:25.193758 29785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:25.193770 29786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:25.194994 29777 server_base.cc:1034] running on GCE node
I20260809 05:40:25.195978 29777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:25.197489 29777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:25.198740 29777 hybrid_clock.cc:648] HybridClock initialized: now 1786254025198684 us; error 40 us; skew 500 ppm
I20260809 05:40:25.199110 29777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:25 dist-test-slave-3kk6 krb5kdc[29759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254025, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:25.202507 29777 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:25.204149 29777 webserver.cc:460] Webserver started at http://127.0.0.1:38205/ using document root <none> and password file <none>
I20260809 05:40:25.204880 29777 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:25.205034 29777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:25.205366 29777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:25.208567 29777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "7cb8785667de4d8eabcfa66682423ae2"
format_stamp: "Formatted at 2026-08-09 05:40:25 on dist-test-slave-3kk6"
server_key: "7f3d22e95bd934ad18e29303530eb55f"
server_key_iv: "1fafd885e29a9d86272eef4176bb334c"
server_key_version: "encryptionkey@0"
I20260809 05:40:25.209417 29777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "7cb8785667de4d8eabcfa66682423ae2"
format_stamp: "Formatted at 2026-08-09 05:40:25 on dist-test-slave-3kk6"
server_key: "7f3d22e95bd934ad18e29303530eb55f"
server_key_iv: "1fafd885e29a9d86272eef4176bb334c"
server_key_version: "encryptionkey@0"
I20260809 05:40:25.214943 29777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:40:25.218626 29799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:25.220052 29777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:40:25.220340 29777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "7cb8785667de4d8eabcfa66682423ae2"
format_stamp: "Formatted at 2026-08-09 05:40:25 on dist-test-slave-3kk6"
server_key: "7f3d22e95bd934ad18e29303530eb55f"
server_key_iv: "1fafd885e29a9d86272eef4176bb334c"
server_key_version: "encryptionkey@0"
I20260809 05:40:25.220551 29777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:25.244211 29777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:25.248785 29777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:25.249104 29777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:25.268877 29777 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32821
I20260809 05:40:25.268896 29878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32821 every 8 connection(s)
I20260809 05:40:25.270314 29777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:25.274057 29879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:25.279032 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 29777
I20260809 05:40:25.279243 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:25.279829 31271 external_mini_cluster.cc:1428] Setting key 551708c371f31e8732c8b92979249f75
I20260809 05:40:25.285981 29879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2: Bootstrap starting.
I20260809 05:40:25.289371 29879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:25.290788 29879 log.cc:824] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:25.294401 29879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2: No bootstrap required, opened a new log
I20260809 05:40:25.295091 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:25.281778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37601 (local address 127.0.0.1:32821)
0809 05:40:25.282643 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:25.282707 (+ 64us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:25.282755 (+ 48us) server_negotiation.cc:407] Connection header received
0809 05:40:25.284638 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:25.284660 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:25.285047 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:25.285429 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:25.285955 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:25.287812 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:25.289153 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:25.290200 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:25.290396 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:25.292278 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:25.292286 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:25.292300 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:25.293022 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:25.293116 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:25.293149 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:25.293282 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:25.293475 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 05:40:25.293742 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":420,"threads_started":1}
I20260809 05:40:25.299047 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:25.302896 29879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8785667de4d8eabcfa66682423ae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 05:40:25.303325 29879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:25.303503 29879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb8785667de4d8eabcfa66682423ae2, State: Initialized, Role: FOLLOWER
I20260809 05:40:25.304288 29879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8785667de4d8eabcfa66682423ae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 05:40:25.304616 29879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:25.304788 29879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:25.305011 29879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:25.308043 29879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8785667de4d8eabcfa66682423ae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 05:40:25.308730 29879 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb8785667de4d8eabcfa66682423ae2; no voters:
I20260809 05:40:25.309494 29879 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:25.309614 29895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:25.311075 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:25.303678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53599 (local address 127.0.0.1:32821)
0809 05:40:25.303884 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:25.303892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:25.304306 (+ 414us) server_negotiation.cc:407] Connection header received
0809 05:40:25.304474 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:25.304481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:25.304568 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:25.304693 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:25.305331 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:25.306467 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:25.307972 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:25.308638 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:25.308796 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:25.310104 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:25.310117 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:25.310122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:25.310149 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:25.310208 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:25.310239 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:25.310310 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:25.310558 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 05:40:25.310679 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 05:40:25.311013 29895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [term 1 LEADER]: Becoming Leader. State: Replica: 7cb8785667de4d8eabcfa66682423ae2, State: Running, Role: LEADER
I20260809 05:40:25.311636 29895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8785667de4d8eabcfa66682423ae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260809 05:40:25.312182 29879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:25.314842 29899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb8785667de4d8eabcfa66682423ae2. Latest consensus state: current_term: 1 leader_uuid: "7cb8785667de4d8eabcfa66682423ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8785667de4d8eabcfa66682423ae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260809 05:40:25.314881 29896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb8785667de4d8eabcfa66682423ae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb8785667de4d8eabcfa66682423ae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260809 05:40:25.315400 29899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:25.315410 29896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:25.317075 29902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:25.322379 29902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:25.332536 29902 catalog_manager.cc:1347] Generated new cluster ID: f27e979c8d2d455981bb0a455fdc48b2
I20260809 05:40:25.332664 29902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:25.343173 29902 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:25.344517 29902 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:25.351199 29902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cb8785667de4d8eabcfa66682423ae2: Generated new TSK 0
I20260809 05:40:25.352090 29902 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:25.357899 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29777
W20260809 05:40:25.359509 29871 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37601
W20260809 05:40:25.360386 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29777: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:25.360538 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 29777
2026-08-09T05:40:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:25 dist-test-slave-3kk6 krb5kdc[29926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:25 dist-test-slave-3kk6 krb5kdc[29926](info): set up 2 sockets
Aug 09 05:40:25 dist-test-slave-3kk6 krb5kdc[29926](info): commencing operation
krb5kdc: starting...
W20260809 05:40:26.695043 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.002s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:26 dist-test-slave-3kk6 krb5kdc[29926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:27.046285 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:27.404914 29947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:27.405226 29947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:27.405321 29947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:27.413303 29947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:27.413432 29947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:27.413479 29947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:27.413543 29947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:27.413589 29947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:27.423748 29947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:27.425082 29947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:27.426435 29947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:27.438884 29956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:27.438949 29957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:27.439013 29959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:27.439803 29947 server_base.cc:1034] running on GCE node
I20260809 05:40:27.440558 29947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:27.441938 29947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:27.443182 29947 hybrid_clock.cc:648] HybridClock initialized: now 1786254027443141 us; error 51 us; skew 500 ppm
I20260809 05:40:27.443567 29947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:27 dist-test-slave-3kk6 krb5kdc[29926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254027, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:27.452085 29947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:27.453816 29947 webserver.cc:460] Webserver started at http://127.0.0.1:43253/ using document root <none> and password file <none>
I20260809 05:40:27.454581 29947 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:27.454772 29947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:27.455078 29947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:27.458222 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "06b299b8c8d74a34a4b3f95259b8185b"
format_stamp: "Formatted at 2026-08-09 05:40:27 on dist-test-slave-3kk6"
server_key: "2fe9e21c587cc9bd4b7647d5d24b8769"
server_key_iv: "27031cff2c3ebe4042771d113edaab08"
server_key_version: "encryptionkey@0"
I20260809 05:40:27.459102 29947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "06b299b8c8d74a34a4b3f95259b8185b"
format_stamp: "Formatted at 2026-08-09 05:40:27 on dist-test-slave-3kk6"
server_key: "2fe9e21c587cc9bd4b7647d5d24b8769"
server_key_iv: "27031cff2c3ebe4042771d113edaab08"
server_key_version: "encryptionkey@0"
I20260809 05:40:27.464566 29947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:40:27.468218 29967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:27.469637 29947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 05:40:27.469799 29947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "06b299b8c8d74a34a4b3f95259b8185b"
format_stamp: "Formatted at 2026-08-09 05:40:27 on dist-test-slave-3kk6"
server_key: "2fe9e21c587cc9bd4b7647d5d24b8769"
server_key_iv: "27031cff2c3ebe4042771d113edaab08"
server_key_version: "encryptionkey@0"
I20260809 05:40:27.469992 29947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:27.476153 29947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:27.480907 29947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:27.481171 29947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:27.500895 29947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39531
I20260809 05:40:27.500967 30045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39531 every 8 connection(s)
I20260809 05:40:27.502488 29947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:27.506311 30046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:27.511591 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 29947
I20260809 05:40:27.511818 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:27.512349 31271 external_mini_cluster.cc:1428] Setting key 05c3c8367256e397615c6dfff861ad43
I20260809 05:40:27.518234 30046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b: Bootstrap starting.
I20260809 05:40:27.520867 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:27.514218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32891 (local address 127.0.0.1:39531)
0809 05:40:27.515144 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:27.515157 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:27.515205 (+ 48us) server_negotiation.cc:407] Connection header received
0809 05:40:27.517233 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:27.517263 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:27.517618 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:27.517996 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:27.519252 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:27.519258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:27.519270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:27.519283 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:27.519333 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:27.519351 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:27.519522 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:27.519681 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 05:40:27.519907 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":480,"threads_started":1}
I20260809 05:40:27.522125 30046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:27.523387 30046 log.cc:824] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:27.524746 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:27.526337 30046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b: No bootstrap required, opened a new log
I20260809 05:40:27.532389 30050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:27.529511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37129 (local address 127.0.0.1:39531)
0809 05:40:27.529727 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:27.529734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:27.530104 (+ 370us) server_negotiation.cc:407] Connection header received
0809 05:40:27.530387 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:27.530390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:27.530442 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:27.530533 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:27.531685 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:27.531689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:27.531692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:27.531707 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:27.531749 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:27.531767 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:27.531858 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:27.531997 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 05:40:27.532045 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 05:40:27.535805 30046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b299b8c8d74a34a4b3f95259b8185b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39531 } }
I20260809 05:40:27.536206 30046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:27.536342 30046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b299b8c8d74a34a4b3f95259b8185b, State: Initialized, Role: FOLLOWER
I20260809 05:40:27.536928 30046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b299b8c8d74a34a4b3f95259b8185b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39531 } }
I20260809 05:40:27.537179 30046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:27.537348 30046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:27.537549 30046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:27.540154 30046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b299b8c8d74a34a4b3f95259b8185b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39531 } }
I20260809 05:40:27.540746 30046 leader_election.cc:302] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06b299b8c8d74a34a4b3f95259b8185b; no voters:
I20260809 05:40:27.541393 30046 leader_election.cc:288] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:27.541532 30059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:27.542894 30059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [term 1 LEADER]: Becoming Leader. State: Replica: 06b299b8c8d74a34a4b3f95259b8185b, State: Running, Role: LEADER
I20260809 05:40:27.543465 30059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b299b8c8d74a34a4b3f95259b8185b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39531 } }
I20260809 05:40:27.543833 30046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:27.546478 30061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06b299b8c8d74a34a4b3f95259b8185b. Latest consensus state: current_term: 1 leader_uuid: "06b299b8c8d74a34a4b3f95259b8185b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b299b8c8d74a34a4b3f95259b8185b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39531 } } }
I20260809 05:40:27.546986 30061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:27.547564 30060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06b299b8c8d74a34a4b3f95259b8185b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06b299b8c8d74a34a4b3f95259b8185b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39531 } } }
I20260809 05:40:27.547889 30060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:27.549811 30070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:27.555276 30070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:27.565964 30070 catalog_manager.cc:1347] Generated new cluster ID: 0f4fa6f91ec243c792047d8c74f8b8c7
I20260809 05:40:27.566112 30070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:27.578910 30070 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:27.579413 30070 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:27.588575 30070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06b299b8c8d74a34a4b3f95259b8185b: Generated new TSK 0
I20260809 05:40:27.589690 30070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:27.600104 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29947
W20260809 05:40:27.601327 30040 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32891
W20260809 05:40:27.602078 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 29947: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:27.602241 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 29947
2026-08-09T05:40:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:27 dist-test-slave-3kk6 krb5kdc[30089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:27 dist-test-slave-3kk6 krb5kdc[30089](info): set up 2 sockets
Aug 09 05:40:27 dist-test-slave-3kk6 krb5kdc[30089](info): commencing operation
krb5kdc: starting...
W20260809 05:40:28.955282 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.005s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:29 dist-test-slave-3kk6 krb5kdc[30089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:29.321559 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:29.685438 30108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:29.685856 30108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:29.686029 30108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:29.693588 30108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:29.693787 30108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:29.693907 30108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:29.694024 30108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:29.694103 30108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:29.704773 30108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:29.706276 30108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:29.707736 30108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:29.719967 30117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:29.720191 30120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:29.720016 30116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:29.720862 30108 server_base.cc:1034] running on GCE node
I20260809 05:40:29.721740 30108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:29.723292 30108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:29.724509 30108 hybrid_clock.cc:648] HybridClock initialized: now 1786254029724444 us; error 57 us; skew 500 ppm
I20260809 05:40:29.724916 30108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:29 dist-test-slave-3kk6 krb5kdc[30089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:29.728332 30108 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:29.729805 30108 webserver.cc:460] Webserver started at http://127.0.0.1:35643/ using document root <none> and password file <none>
I20260809 05:40:29.730453 30108 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:29.730556 30108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:29.730890 30108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:29.733986 30108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "5d485c95940d42d88ae8ec3b90a69137"
format_stamp: "Formatted at 2026-08-09 05:40:29 on dist-test-slave-3kk6"
server_key: "62a2acce95aef085358edf6d95db64af"
server_key_iv: "fd79d4050a0587be6d8c89ec63f4171c"
server_key_version: "encryptionkey@0"
I20260809 05:40:29.734915 30108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "5d485c95940d42d88ae8ec3b90a69137"
format_stamp: "Formatted at 2026-08-09 05:40:29 on dist-test-slave-3kk6"
server_key: "62a2acce95aef085358edf6d95db64af"
server_key_iv: "fd79d4050a0587be6d8c89ec63f4171c"
server_key_version: "encryptionkey@0"
I20260809 05:40:29.740278 30108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 05:40:29.743782 30133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:29.745141 30108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:40:29.745332 30108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "5d485c95940d42d88ae8ec3b90a69137"
format_stamp: "Formatted at 2026-08-09 05:40:29 on dist-test-slave-3kk6"
server_key: "62a2acce95aef085358edf6d95db64af"
server_key_iv: "fd79d4050a0587be6d8c89ec63f4171c"
server_key_version: "encryptionkey@0"
I20260809 05:40:29.745530 30108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:29.769218 30108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:29.773826 30108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:29.774166 30108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:29.794299 30108 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33863
I20260809 05:40:29.794339 30207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33863 every 8 connection(s)
I20260809 05:40:29.795835 30108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:29.798381 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 30108
I20260809 05:40:29.798592 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:29.799192 31271 external_mini_cluster.cc:1428] Setting key 488886e4bf84daaf1fa4f547bff14e85
I20260809 05:40:29.800326 30208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:40:29 dist-test-slave-3kk6 krb5kdc[30089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:40:29.815212 30208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137: Bootstrap starting.
I20260809 05:40:29.819103 30208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:29.820374 30208 log.cc:824] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:29.823671 30208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137: No bootstrap required, opened a new log
I20260809 05:40:29.824545 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:29.801229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:60379 (local address 127.0.0.1:33863)
0809 05:40:29.802033 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:29.802051 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:29.802106 (+ 55us) server_negotiation.cc:407] Connection header received
0809 05:40:29.804165 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:29.804193 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:29.804759 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:29.805240 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:29.806627 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:29.808798 (+ 2171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:29.810122 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:29.811238 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:29.811444 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:29.816136 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:29.816144 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:29.816164 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:29.816202 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:29.819813 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:29.820859 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:29.820866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:29.820872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:29.821078 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:29.821630 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:29.821634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:29.821637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:29.822201 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:29.822549 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:29.823127 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 05:40:29.823481 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":314,"threads_started":1}
I20260809 05:40:29.828912 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:29.832968 30208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d485c95940d42d88ae8ec3b90a69137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260809 05:40:29.833527 30208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:29.833693 30208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d485c95940d42d88ae8ec3b90a69137, State: Initialized, Role: FOLLOWER
I20260809 05:40:29.834455 30208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d485c95940d42d88ae8ec3b90a69137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260809 05:40:29.834766 30208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:29.834947 30208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:29.835145 30208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:29.837549 30208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d485c95940d42d88ae8ec3b90a69137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260809 05:40:29.838106 30208 leader_election.cc:302] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d485c95940d42d88ae8ec3b90a69137; no voters:
I20260809 05:40:29.838901 30208 leader_election.cc:288] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:29.839033 30226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:29.840269 30226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [term 1 LEADER]: Becoming Leader. State: Replica: 5d485c95940d42d88ae8ec3b90a69137, State: Running, Role: LEADER
I20260809 05:40:29.841324 30226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d485c95940d42d88ae8ec3b90a69137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260809 05:40:29.841573 30208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:29.845034 30227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d485c95940d42d88ae8ec3b90a69137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d485c95940d42d88ae8ec3b90a69137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } } }
I20260809 05:40:29.845932 30227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:29.846872 30228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d485c95940d42d88ae8ec3b90a69137. Latest consensus state: current_term: 1 leader_uuid: "5d485c95940d42d88ae8ec3b90a69137" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d485c95940d42d88ae8ec3b90a69137" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } } }
I20260809 05:40:29.847266 30228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:29.848790 30233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:29.854226 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:29.833834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:47193 (local address 127.0.0.1:33863)
0809 05:40:29.834178 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:29.834185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:29.834471 (+ 286us) server_negotiation.cc:407] Connection header received
0809 05:40:29.834700 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:29.834706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:29.834782 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:29.834892 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:29.836464 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:29.837280 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:29.838788 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:29.839505 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:29.839668 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:29.842904 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:29.842912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:29.842919 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:29.842975 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:29.847406 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:29.848381 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:29.848387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:29.848391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:29.848593 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:29.849258 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:29.849265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:29.849269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:29.849585 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:29.849953 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:29.850894 (+ 941us) server_negotiation.cc:299] Negotiation successful
0809 05:40:29.851028 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 05:40:29.859028 30233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:29.870419 30233 catalog_manager.cc:1347] Generated new cluster ID: 7535a5928a784f9daec4c189ce3ea713
I20260809 05:40:29.870568 30233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:29.879959 30233 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:29.881767 30233 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:29.890959 30233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d485c95940d42d88ae8ec3b90a69137: Generated new TSK 0
I20260809 05:40:29.892190 30233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:29.905985 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30108
I20260809 05:40:29.907254 30201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:30.001865 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 30108
W20260809 05:40:30.027644 30087 connection.cc:535] client connection to 127.0.0.1:33863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33863): Connection reset by peer (error 104)
2026-08-09T05:40:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:30 dist-test-slave-3kk6 krb5kdc[30261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:30 dist-test-slave-3kk6 krb5kdc[30261](info): set up 2 sockets
Aug 09 05:40:30 dist-test-slave-3kk6 krb5kdc[30261](info): commencing operation
krb5kdc: starting...
W20260809 05:40:31.350863 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:31 dist-test-slave-3kk6 krb5kdc[30261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:31.715497 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:32.073969 30282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:32.074391 30282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:32.074581 30282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:32.082551 30282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:32.082788 30282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:32.082898 30282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:32.083017 30282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:32.083108 30282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:32.094718 30282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:32.096211 30282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:32.098549 30282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:32.112615 30293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:32.113265 30294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:32.113529 30296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:32.114580 30282 server_base.cc:1034] running on GCE node
I20260809 05:40:32.115417 30282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:32.117306 30282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:32.118508 30282 hybrid_clock.cc:648] HybridClock initialized: now 1786254032118460 us; error 69 us; skew 500 ppm
I20260809 05:40:32.118906 30282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:32 dist-test-slave-3kk6 krb5kdc[30261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:32.122190 30282 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:32.123706 30282 webserver.cc:460] Webserver started at http://127.0.0.1:35659/ using document root <none> and password file <none>
I20260809 05:40:32.124450 30282 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:32.124608 30282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:32.124891 30282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:32.127938 30282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "ed56f3573f5c4ab79910aeb577d7b436"
format_stamp: "Formatted at 2026-08-09 05:40:32 on dist-test-slave-3kk6"
server_key: "04e69f6f960104dda71b057e1a4ca7c8"
server_key_iv: "89e1eddc84407151d5e30926ee750990"
server_key_version: "encryptionkey@0"
I20260809 05:40:32.128670 30282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "ed56f3573f5c4ab79910aeb577d7b436"
format_stamp: "Formatted at 2026-08-09 05:40:32 on dist-test-slave-3kk6"
server_key: "04e69f6f960104dda71b057e1a4ca7c8"
server_key_iv: "89e1eddc84407151d5e30926ee750990"
server_key_version: "encryptionkey@0"
I20260809 05:40:32.133939 30282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:40:32.137676 30304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:32.139242 30282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:32.139447 30282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "ed56f3573f5c4ab79910aeb577d7b436"
format_stamp: "Formatted at 2026-08-09 05:40:32 on dist-test-slave-3kk6"
server_key: "04e69f6f960104dda71b057e1a4ca7c8"
server_key_iv: "89e1eddc84407151d5e30926ee750990"
server_key_version: "encryptionkey@0"
I20260809 05:40:32.139657 30282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:32.152384 30282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:32.157123 30282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:32.157400 30282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:32.177969 30282 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41775
I20260809 05:40:32.178017 30390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41775 every 8 connection(s)
I20260809 05:40:32.179535 30282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:32.180605 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 30282
I20260809 05:40:32.180819 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:32.181313 31271 external_mini_cluster.cc:1428] Setting key 2eccb545bc2b2ef78d312f5430668de2
I20260809 05:40:32.184293 30391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:32.194960 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:32.183292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:47141 (local address 127.0.0.1:41775)
0809 05:40:32.184091 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:32.184102 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:32.184146 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:40:32.185898 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:32.185920 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:32.186320 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:32.186762 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:32.187428 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:32.188767 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:32.190134 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:32.190905 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:32.191106 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:32.192504 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:32.192512 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:32.192526 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:32.193118 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:32.193198 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:32.193229 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:32.193421 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:32.193713 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 05:40:32.193885 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":345,"threads_started":1}
I20260809 05:40:32.195916 30391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436: Bootstrap starting.
I20260809 05:40:32.199182 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:32.199865 30391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:32.200984 30391 log.cc:824] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:32.203959 30391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436: No bootstrap required, opened a new log
I20260809 05:40:32.211336 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:32.204166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:47883 (local address 127.0.0.1:41775)
0809 05:40:32.204353 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:32.204359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:32.204855 (+ 496us) server_negotiation.cc:407] Connection header received
0809 05:40:32.205014 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:32.205019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:32.205099 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:32.205218 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:32.205733 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:32.207018 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:32.208455 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:32.209177 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:32.209327 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:32.210488 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:32.210496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:32.210501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:32.210528 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:32.210574 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:32.210600 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:32.210764 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:32.210926 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 05:40:32.210979 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 05:40:32.214180 30391 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed56f3573f5c4ab79910aeb577d7b436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41775 } }
I20260809 05:40:32.214712 30391 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:32.214861 30391 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed56f3573f5c4ab79910aeb577d7b436, State: Initialized, Role: FOLLOWER
I20260809 05:40:32.215579 30391 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed56f3573f5c4ab79910aeb577d7b436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41775 } }
I20260809 05:40:32.215818 30391 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:32.215925 30391 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:32.216060 30391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:32.218254 30391 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed56f3573f5c4ab79910aeb577d7b436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41775 } }
I20260809 05:40:32.218784 30391 leader_election.cc:302] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed56f3573f5c4ab79910aeb577d7b436; no voters:
I20260809 05:40:32.219326 30391 leader_election.cc:288] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:32.219493 30405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:32.220247 30405 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [term 1 LEADER]: Becoming Leader. State: Replica: ed56f3573f5c4ab79910aeb577d7b436, State: Running, Role: LEADER
I20260809 05:40:32.220777 30405 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed56f3573f5c4ab79910aeb577d7b436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41775 } }
I20260809 05:40:32.221685 30391 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:32.223902 30408 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed56f3573f5c4ab79910aeb577d7b436. Latest consensus state: current_term: 1 leader_uuid: "ed56f3573f5c4ab79910aeb577d7b436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed56f3573f5c4ab79910aeb577d7b436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41775 } } }
I20260809 05:40:32.224061 30407 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed56f3573f5c4ab79910aeb577d7b436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed56f3573f5c4ab79910aeb577d7b436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41775 } } }
I20260809 05:40:32.224498 30408 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:32.224732 30407 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:32.227694 30423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:32.232924 30423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:32.244011 30423 catalog_manager.cc:1347] Generated new cluster ID: e8b900f6a97a410ea89e6a7a303057dd
I20260809 05:40:32.244174 30423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:32.256119 30423 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:32.257475 30423 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:32.269239 30423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed56f3573f5c4ab79910aeb577d7b436: Generated new TSK 0
I20260809 05:40:32.270170 30423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:32.286487 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30282
W20260809 05:40:32.287745 30382 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.97:47141
W20260809 05:40:32.288626 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30282: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:32.288807 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 30282
W20260809 05:40:32.319981 30259 connection.cc:535] client connection to 127.0.0.1:41775 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41775): Connection reset by peer (error 104)
2026-08-09T05:40:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:32 dist-test-slave-3kk6 krb5kdc[30440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:32 dist-test-slave-3kk6 krb5kdc[30440](info): set up 2 sockets
Aug 09 05:40:32 dist-test-slave-3kk6 krb5kdc[30440](info): commencing operation
krb5kdc: starting...
W20260809 05:40:33.672290 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:33 dist-test-slave-3kk6 krb5kdc[30440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:34.041779 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:34.418319 30460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:34.418738 30460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:34.418888 30460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:34.426470 30460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:34.426620 30460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:34.426753 30460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:34.426872 30460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:34.426962 30460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:34.437556 30460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:34.439169 30460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:34.440662 30460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:34.454186 30471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:34.454458 30469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:34.454420 30473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:34.460412 30460 server_base.cc:1034] running on GCE node
I20260809 05:40:34.461475 30460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:34.463246 30460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:34.464490 30460 hybrid_clock.cc:648] HybridClock initialized: now 1786254034464439 us; error 44 us; skew 500 ppm
I20260809 05:40:34.464942 30460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:34 dist-test-slave-3kk6 krb5kdc[30440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:34.472002 30460 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:34.473874 30460 webserver.cc:460] Webserver started at http://127.0.0.1:39587/ using document root <none> and password file <none>
I20260809 05:40:34.474617 30460 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:34.474831 30460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:34.475198 30460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:34.478474 30460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "bd53a7af656542548c1313df7039e341"
format_stamp: "Formatted at 2026-08-09 05:40:34 on dist-test-slave-3kk6"
server_key: "6c98707a78780e19e58916d562a1294c"
server_key_iv: "61e29569c3cd602de0e7cf6a09dd854e"
server_key_version: "encryptionkey@0"
I20260809 05:40:34.479487 30460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "bd53a7af656542548c1313df7039e341"
format_stamp: "Formatted at 2026-08-09 05:40:34 on dist-test-slave-3kk6"
server_key: "6c98707a78780e19e58916d562a1294c"
server_key_iv: "61e29569c3cd602de0e7cf6a09dd854e"
server_key_version: "encryptionkey@0"
I20260809 05:40:34.485383 30460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:34.490249 30483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:34.491866 30460 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 05:40:34.492090 30460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "bd53a7af656542548c1313df7039e341"
format_stamp: "Formatted at 2026-08-09 05:40:34 on dist-test-slave-3kk6"
server_key: "6c98707a78780e19e58916d562a1294c"
server_key_iv: "61e29569c3cd602de0e7cf6a09dd854e"
server_key_version: "encryptionkey@0"
I20260809 05:40:34.492331 30460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:34.499006 30460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:34.503790 30460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:34.504139 30460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:34.524734 30460 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43313
I20260809 05:40:34.524772 30562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43313 every 8 connection(s)
I20260809 05:40:34.526324 30460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:34.529219 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 30460
I20260809 05:40:34.529443 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:34.530022 31271 external_mini_cluster.cc:1428] Setting key 46b25a5052522433cfa33cff488b0366
I20260809 05:40:34.531093 30564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:34.539019 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:34.532094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:54071 (local address 127.0.0.1:43313)
0809 05:40:34.532991 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:34.533004 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:34.533050 (+ 46us) server_negotiation.cc:407] Connection header received
0809 05:40:34.534833 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:34.534860 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:34.535290 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:34.535765 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:34.537120 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:34.537127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:34.537141 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:34.537158 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:34.537248 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:34.537269 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:34.537540 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:34.537699 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 05:40:34.537909 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":374,"threads_started":1}
I20260809 05:40:34.543466 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:34.548453 30564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341: Bootstrap starting.
I20260809 05:40:34.551743 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:34.548531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:49089 (local address 127.0.0.1:43313)
0809 05:40:34.548724 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:34.548730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:34.549322 (+ 592us) server_negotiation.cc:407] Connection header received
0809 05:40:34.549573 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:34.549577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:34.549631 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:34.549734 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:34.550992 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:34.550997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:34.551001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:34.551018 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:34.551059 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:34.551087 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:34.551225 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:34.551349 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 05:40:34.551419 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 05:40:34.553815 30564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:34.555339 30564 log.cc:824] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:34.558583 30564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341: No bootstrap required, opened a new log
I20260809 05:40:34.566907 30564 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd53a7af656542548c1313df7039e341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43313 } }
I20260809 05:40:34.567360 30564 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:34.567500 30564 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd53a7af656542548c1313df7039e341, State: Initialized, Role: FOLLOWER
I20260809 05:40:34.568130 30564 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd53a7af656542548c1313df7039e341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43313 } }
I20260809 05:40:34.568382 30564 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:34.568557 30564 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:34.568717 30564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:34.571614 30564 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd53a7af656542548c1313df7039e341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43313 } }
I20260809 05:40:34.572266 30564 leader_election.cc:302] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd53a7af656542548c1313df7039e341; no voters:
I20260809 05:40:34.573022 30564 leader_election.cc:288] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:34.573179 30576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:34.574138 30576 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [term 1 LEADER]: Becoming Leader. State: Replica: bd53a7af656542548c1313df7039e341, State: Running, Role: LEADER
I20260809 05:40:34.574863 30576 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd53a7af656542548c1313df7039e341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43313 } }
I20260809 05:40:34.575454 30564 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:34.578068 30580 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd53a7af656542548c1313df7039e341. Latest consensus state: current_term: 1 leader_uuid: "bd53a7af656542548c1313df7039e341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd53a7af656542548c1313df7039e341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43313 } } }
I20260809 05:40:34.578552 30580 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:34.578984 30578 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd53a7af656542548c1313df7039e341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd53a7af656542548c1313df7039e341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43313 } } }
I20260809 05:40:34.579456 30578 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:34.587987 30588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:34.594017 30588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:34.603758 30588 catalog_manager.cc:1347] Generated new cluster ID: 5ff87bcf156847ce969154dfa28407b8
I20260809 05:40:34.603911 30588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:34.630004 30588 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:34.630584 30588 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:34.641952 30588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd53a7af656542548c1313df7039e341: Generated new TSK 0
I20260809 05:40:34.643169 30588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:34.652424 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30460
W20260809 05:40:34.653777 30555 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.97:54071
W20260809 05:40:34.654583 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30460: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:34.654790 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 30460
2026-08-09T05:40:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:34 dist-test-slave-3kk6 krb5kdc[30610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:34 dist-test-slave-3kk6 krb5kdc[30610](info): set up 2 sockets
Aug 09 05:40:34 dist-test-slave-3kk6 krb5kdc[30610](info): commencing operation
krb5kdc: starting...
W20260809 05:40:36.009768 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:36 dist-test-slave-3kk6 krb5kdc[30610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:36.375550 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:36.737350 30627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:36.737665 30627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:36.737845 30627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:36.745361 30627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:36.745496 30627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:36.745620 30627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:36.745724 30627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:36.745816 30627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:36.756289 30627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:36.757673 30627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:36.759196 30627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:36.771916 30644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:36.771874 30636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:36.771966 30637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:36.776219 30627 server_base.cc:1034] running on GCE node
I20260809 05:40:36.777127 30627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:36.778728 30627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:36.779984 30627 hybrid_clock.cc:648] HybridClock initialized: now 1786254036779938 us; error 38 us; skew 500 ppm
I20260809 05:40:36.780385 30627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:36 dist-test-slave-3kk6 krb5kdc[30610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:36.783722 30627 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:36.785351 30627 webserver.cc:460] Webserver started at http://127.0.0.1:39153/ using document root <none> and password file <none>
I20260809 05:40:36.786073 30627 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:36.786232 30627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:36.786564 30627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:36.789646 30627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "ca5081c5e3374e93a03ff090bad8673b"
format_stamp: "Formatted at 2026-08-09 05:40:36 on dist-test-slave-3kk6"
server_key: "426adc24bcd803659de57e7e643b0d4b"
server_key_iv: "4202aa39bb7d5c85cb48f4101ed226fe"
server_key_version: "encryptionkey@0"
I20260809 05:40:36.790488 30627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "ca5081c5e3374e93a03ff090bad8673b"
format_stamp: "Formatted at 2026-08-09 05:40:36 on dist-test-slave-3kk6"
server_key: "426adc24bcd803659de57e7e643b0d4b"
server_key_iv: "4202aa39bb7d5c85cb48f4101ed226fe"
server_key_version: "encryptionkey@0"
I20260809 05:40:36.796001 30627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 05:40:36.799647 30654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:36.801100 30627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 05:40:36.801307 30627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "ca5081c5e3374e93a03ff090bad8673b"
format_stamp: "Formatted at 2026-08-09 05:40:36 on dist-test-slave-3kk6"
server_key: "426adc24bcd803659de57e7e643b0d4b"
server_key_iv: "4202aa39bb7d5c85cb48f4101ed226fe"
server_key_version: "encryptionkey@0"
I20260809 05:40:36.801522 30627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:36.823741 30627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:36.828137 30627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:36.828460 30627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:36.848343 30627 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38941
I20260809 05:40:36.848393 30741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38941 every 8 connection(s)
I20260809 05:40:36.849786 30627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:36.851274 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 30627
I20260809 05:40:36.851482 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:36.852134 31271 external_mini_cluster.cc:1428] Setting key 6840f60e96f2294fb7cf54544e112761
I20260809 05:40:36.854869 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:40:36 dist-test-slave-3kk6 krb5kdc[30610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 05:40:36.869441 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b: Bootstrap starting.
I20260809 05:40:36.873080 30742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:36.874224 30742 log.cc:824] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:36.877396 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b: No bootstrap required, opened a new log
I20260809 05:40:36.877707 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:36.854073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:47799 (local address 127.0.0.1:38941)
0809 05:40:36.854973 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:36.854991 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:36.855053 (+ 62us) server_negotiation.cc:407] Connection header received
0809 05:40:36.856748 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:36.856771 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:36.857167 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:36.857607 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:36.859706 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:36.861171 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:36.863140 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:36.863895 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:36.864059 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:36.869612 (+ 5553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:36.869625 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:36.869648 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:36.869705 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:36.873068 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:36.874013 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:36.874021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:36.874027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:36.874224 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:36.874813 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:36.874817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:36.874819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:36.875399 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:36.875750 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:36.876312 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 05:40:36.876651 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":370,"threads_started":1}
I20260809 05:40:36.881732 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:36.885762 30742 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca5081c5e3374e93a03ff090bad8673b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38941 } }
I20260809 05:40:36.886266 30742 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:36.886415 30742 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5081c5e3374e93a03ff090bad8673b, State: Initialized, Role: FOLLOWER
I20260809 05:40:36.887085 30742 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca5081c5e3374e93a03ff090bad8673b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38941 } }
I20260809 05:40:36.887398 30742 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:36.887573 30742 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:36.887777 30742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:36.890625 30742 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca5081c5e3374e93a03ff090bad8673b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38941 } }
I20260809 05:40:36.891228 30742 leader_election.cc:302] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca5081c5e3374e93a03ff090bad8673b; no voters:
I20260809 05:40:36.891893 30742 leader_election.cc:288] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:36.892016 30761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:36.893975 30742 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:36.899058 30761 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [term 1 LEADER]: Becoming Leader. State: Replica: ca5081c5e3374e93a03ff090bad8673b, State: Running, Role: LEADER
I20260809 05:40:36.900360 30761 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca5081c5e3374e93a03ff090bad8673b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38941 } }
I20260809 05:40:36.903587 30766 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca5081c5e3374e93a03ff090bad8673b. Latest consensus state: current_term: 1 leader_uuid: "ca5081c5e3374e93a03ff090bad8673b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca5081c5e3374e93a03ff090bad8673b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38941 } } }
I20260809 05:40:36.904114 30766 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:36.903937 30765 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca5081c5e3374e93a03ff090bad8673b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca5081c5e3374e93a03ff090bad8673b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38941 } } }
I20260809 05:40:36.904448 30765 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:36.906494 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:36.886238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:54613 (local address 127.0.0.1:38941)
0809 05:40:36.886447 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:36.886453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:36.887134 (+ 681us) server_negotiation.cc:407] Connection header received
0809 05:40:36.887284 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:36.887290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:36.887377 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:36.887504 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:36.889146 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:36.890318 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:36.892145 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:36.892901 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:36.893028 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:36.897048 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:36.897055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:36.897060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:36.897098 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:36.901828 (+ 4730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:36.902995 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:36.903002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:36.903005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:36.903265 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:36.903926 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:36.903933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:36.903936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:36.904519 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:36.904934 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:36.905765 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 05:40:36.905974 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 05:40:36.910873 30784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:40:36.911032 30784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:40:36.918138 30788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:36.919956 30788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:36.929034 30788 catalog_manager.cc:1347] Generated new cluster ID: e70e909e495a4f7b86bfb7c6c3780d64
I20260809 05:40:36.929177 30788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:36.952903 30788 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:36.954262 30788 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:36.969547 30788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca5081c5e3374e93a03ff090bad8673b: Generated new TSK 0
I20260809 05:40:36.970575 30788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:36.982162 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30627
I20260809 05:40:36.983557 30735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:37.079663 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 30627
W20260809 05:40:37.104334 30607 connection.cc:535] client connection to 127.0.0.1:38941 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38941): Connection reset by peer (error 104)
2026-08-09T05:40:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:37 dist-test-slave-3kk6 krb5kdc[30797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:37 dist-test-slave-3kk6 krb5kdc[30797](info): set up 2 sockets
Aug 09 05:40:37 dist-test-slave-3kk6 krb5kdc[30797](info): commencing operation
krb5kdc: starting...
W20260809 05:40:38.437610 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:38 dist-test-slave-3kk6 krb5kdc[30797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:38.791718 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:39.156848 30816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:39.157263 30816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:39.157449 30816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:39.165310 30816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:39.165499 30816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:39.165617 30816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:39.165736 30816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:39.165838 30816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:39.176250 30816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:39.177742 30816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:39.179172 30816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:39.191849 30830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:39.191969 30826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:39.191910 30827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:39.193064 30816 server_base.cc:1034] running on GCE node
I20260809 05:40:39.193962 30816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:39.195501 30816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:39.196775 30816 hybrid_clock.cc:648] HybridClock initialized: now 1786254039196723 us; error 46 us; skew 500 ppm
I20260809 05:40:39.197177 30816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:39 dist-test-slave-3kk6 krb5kdc[30797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254039, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:39.200667 30816 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:39.202315 30816 webserver.cc:460] Webserver started at http://127.0.0.1:41845/ using document root <none> and password file <none>
I20260809 05:40:39.203119 30816 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:39.203277 30816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:39.203598 30816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:39.206861 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "6d3cd8747c184c32a27e0db68303c176"
format_stamp: "Formatted at 2026-08-09 05:40:39 on dist-test-slave-3kk6"
server_key: "ee60b78ce1b173ef60a4b1bdfad3a0f7"
server_key_iv: "f435fe3d860d050f0e0c81902cfafdcc"
server_key_version: "encryptionkey@0"
I20260809 05:40:39.207715 30816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "6d3cd8747c184c32a27e0db68303c176"
format_stamp: "Formatted at 2026-08-09 05:40:39 on dist-test-slave-3kk6"
server_key: "ee60b78ce1b173ef60a4b1bdfad3a0f7"
server_key_iv: "f435fe3d860d050f0e0c81902cfafdcc"
server_key_version: "encryptionkey@0"
I20260809 05:40:39.213244 30816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:39.216970 30843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:39.218467 30816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:40:39.218806 30816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "6d3cd8747c184c32a27e0db68303c176"
format_stamp: "Formatted at 2026-08-09 05:40:39 on dist-test-slave-3kk6"
server_key: "ee60b78ce1b173ef60a4b1bdfad3a0f7"
server_key_iv: "f435fe3d860d050f0e0c81902cfafdcc"
server_key_version: "encryptionkey@0"
I20260809 05:40:39.219039 30816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:39.255787 30816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:39.260769 30816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:39.261123 30816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:39.281659 30816 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44619
I20260809 05:40:39.281684 30936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44619 every 8 connection(s)
I20260809 05:40:39.283257 30816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:39.287425 30937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:39.287827 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 30816
I20260809 05:40:39.288041 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:39.288637 31271 external_mini_cluster.cc:1428] Setting key c44a9da6cb9b59c54a8e9b97d0f98add
I20260809 05:40:39.301048 30937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176: Bootstrap starting.
I20260809 05:40:39.303692 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:39.290805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:36129 (local address 127.0.0.1:44619)
0809 05:40:39.291732 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:39.291747 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:39.291792 (+ 45us) server_negotiation.cc:407] Connection header received
0809 05:40:39.294254 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:39.294280 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:39.294830 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:39.295287 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:39.295905 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:39.297218 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:39.298698 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:39.299437 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:39.299649 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:39.301167 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:39.301178 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:39.301197 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:39.301796 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:39.301888 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:39.301915 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:39.302094 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:39.302444 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 05:40:39.302636 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":431,"threads_started":1}
I20260809 05:40:39.304622 30937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:39.305797 30937 log.cc:824] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:39.308099 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:39.309245 30937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176: No bootstrap required, opened a new log
I20260809 05:40:39.319306 30937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3cd8747c184c32a27e0db68303c176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260809 05:40:39.319893 30937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:39.320026 30937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d3cd8747c184c32a27e0db68303c176, State: Initialized, Role: FOLLOWER
I20260809 05:40:39.320504 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:39.312784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:45221 (local address 127.0.0.1:44619)
0809 05:40:39.313029 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:39.313038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:39.313492 (+ 454us) server_negotiation.cc:407] Connection header received
0809 05:40:39.313695 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:39.313699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:39.313774 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:39.313867 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:39.314498 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:39.315379 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:39.317419 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:39.318034 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:39.318137 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:39.319600 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:39.319607 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:39.319614 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:39.319640 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:39.319696 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:39.319719 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:39.319864 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:39.320123 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 05:40:39.320183 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:40:39.320773 30937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3cd8747c184c32a27e0db68303c176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260809 05:40:39.321060 30937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:39.321228 30937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:39.321445 30937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:39.324219 30937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3cd8747c184c32a27e0db68303c176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260809 05:40:39.324783 30937 leader_election.cc:302] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d3cd8747c184c32a27e0db68303c176; no voters:
I20260809 05:40:39.325408 30937 leader_election.cc:288] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:39.325584 30953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:39.326363 30953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [term 1 LEADER]: Becoming Leader. State: Replica: 6d3cd8747c184c32a27e0db68303c176, State: Running, Role: LEADER
I20260809 05:40:39.326961 30953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3cd8747c184c32a27e0db68303c176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260809 05:40:39.327809 30937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:39.330014 30954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d3cd8747c184c32a27e0db68303c176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3cd8747c184c32a27e0db68303c176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } } }
I20260809 05:40:39.330536 30954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:39.332899 30966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:39.332944 30956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d3cd8747c184c32a27e0db68303c176. Latest consensus state: current_term: 1 leader_uuid: "6d3cd8747c184c32a27e0db68303c176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d3cd8747c184c32a27e0db68303c176" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } } }
I20260809 05:40:39.333357 30956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:39.338385 30966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:39.349684 30966 catalog_manager.cc:1347] Generated new cluster ID: 0d72687e836e4ef7bde927e938b85566
I20260809 05:40:39.349840 30966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:39.361007 30966 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:39.362996 30966 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:39.370554 30966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d3cd8747c184c32a27e0db68303c176: Generated new TSK 0
I20260809 05:40:39.371606 30966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:39.385967 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30816
W20260809 05:40:39.387830 30927 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.97:36129
W20260809 05:40:39.388759 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 30816: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:39.388950 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 30816
W20260809 05:40:39.416306 30794 connection.cc:535] client connection to 127.0.0.1:44619 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44619): Connection reset by peer (error 104)
2026-08-09T05:40:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:39 dist-test-slave-3kk6 krb5kdc[30985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:39 dist-test-slave-3kk6 krb5kdc[30985](info): set up 2 sockets
Aug 09 05:40:39 dist-test-slave-3kk6 krb5kdc[30985](info): commencing operation
krb5kdc: starting...
W20260809 05:40:40.747344 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.004s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:40 dist-test-slave-3kk6 krb5kdc[30985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 05:40:41.114380 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:41.486464 31005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:41.486960 31005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:41.487074 31005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:41.494575 31005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:41.494810 31005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:41.494884 31005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:41.494943 31005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:41.494990 31005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:41.505388 31005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:41.506882 31005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:41.508251 31005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:41.520603 31015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:41.520772 31020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:41.520982 31014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:41.521859 31005 server_base.cc:1034] running on GCE node
I20260809 05:40:41.522732 31005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:41.524212 31005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:41.525471 31005 hybrid_clock.cc:648] HybridClock initialized: now 1786254041525435 us; error 67 us; skew 500 ppm
I20260809 05:40:41.525843 31005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:41 dist-test-slave-3kk6 krb5kdc[30985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254041, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:41.529302 31005 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 05:40:41.530910 31005 webserver.cc:460] Webserver started at http://127.0.0.1:33637/ using document root <none> and password file <none>
I20260809 05:40:41.531601 31005 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:41.531723 31005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:41.532006 31005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:41.535310 31005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "03f6c323f90d474d8279a2f2d5692358"
format_stamp: "Formatted at 2026-08-09 05:40:41 on dist-test-slave-3kk6"
server_key: "614959a51dc01ac189e108b31e5fae10"
server_key_iv: "47d3458c91bbf96330072ebed22f2713"
server_key_version: "encryptionkey@0"
I20260809 05:40:41.536082 31005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "03f6c323f90d474d8279a2f2d5692358"
format_stamp: "Formatted at 2026-08-09 05:40:41 on dist-test-slave-3kk6"
server_key: "614959a51dc01ac189e108b31e5fae10"
server_key_iv: "47d3458c91bbf96330072ebed22f2713"
server_key_version: "encryptionkey@0"
I20260809 05:40:41.541826 31005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:40:41.545871 31032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:41.547605 31005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:41.547797 31005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "03f6c323f90d474d8279a2f2d5692358"
format_stamp: "Formatted at 2026-08-09 05:40:41 on dist-test-slave-3kk6"
server_key: "614959a51dc01ac189e108b31e5fae10"
server_key_iv: "47d3458c91bbf96330072ebed22f2713"
server_key_version: "encryptionkey@0"
I20260809 05:40:41.548023 31005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:41.554476 31005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:41.559208 31005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:41.559514 31005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:41.579927 31005 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33737
I20260809 05:40:41.579964 31117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33737 every 8 connection(s)
I20260809 05:40:41.581476 31005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:41.585415 31119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:41.591928 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31005
I20260809 05:40:41.592147 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:41.592757 31271 external_mini_cluster.cc:1428] Setting key 4b63738f37ea30eba3cb22993475843a
I20260809 05:40:41.597491 31119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358: Bootstrap starting.
I20260809 05:40:41.601100 31119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:41.601406 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:41.594950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:33879 (local address 127.0.0.1:33737)
0809 05:40:41.595841 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:41.595854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:41.595898 (+ 44us) server_negotiation.cc:407] Connection header received
0809 05:40:41.597729 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:41.597762 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:41.598121 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:41.598518 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:41.599692 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:41.599701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:41.599715 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:41.599730 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:41.599796 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:41.599819 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:41.600059 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:41.600196 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 05:40:41.600401 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":383,"threads_started":1}
I20260809 05:40:41.602303 31119 log.cc:824] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:41.605482 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:41.605710 31119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358: No bootstrap required, opened a new log
I20260809 05:40:41.613054 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:41.610309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:34109 (local address 127.0.0.1:33737)
0809 05:40:41.610513 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:41.610519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:41.610977 (+ 458us) server_negotiation.cc:407] Connection header received
0809 05:40:41.611128 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:41.611134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:41.611203 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:41.611313 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:41.612352 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:41.612358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:41.612365 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:41.612390 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:41.612438 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:41.612459 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:41.612605 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:41.612661 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 05:40:41.612728 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:40:41.615061 31119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f6c323f90d474d8279a2f2d5692358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33737 } }
I20260809 05:40:41.615458 31119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:41.615567 31119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f6c323f90d474d8279a2f2d5692358, State: Initialized, Role: FOLLOWER
I20260809 05:40:41.616214 31119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f6c323f90d474d8279a2f2d5692358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33737 } }
I20260809 05:40:41.616562 31119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:41.616711 31119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:41.616941 31119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:41.619735 31119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f6c323f90d474d8279a2f2d5692358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33737 } }
I20260809 05:40:41.620265 31119 leader_election.cc:302] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03f6c323f90d474d8279a2f2d5692358; no voters:
I20260809 05:40:41.620890 31119 leader_election.cc:288] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:41.621068 31134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:41.621793 31134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [term 1 LEADER]: Becoming Leader. State: Replica: 03f6c323f90d474d8279a2f2d5692358, State: Running, Role: LEADER
I20260809 05:40:41.622324 31134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f6c323f90d474d8279a2f2d5692358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33737 } }
I20260809 05:40:41.623428 31119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:41.625626 31136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f6c323f90d474d8279a2f2d5692358. Latest consensus state: current_term: 1 leader_uuid: "03f6c323f90d474d8279a2f2d5692358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f6c323f90d474d8279a2f2d5692358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33737 } } }
I20260809 05:40:41.626137 31136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:41.628749 31145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:41.628579 31135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03f6c323f90d474d8279a2f2d5692358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03f6c323f90d474d8279a2f2d5692358" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33737 } } }
I20260809 05:40:41.628892 31135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:41.635226 31145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:41.645232 31145 catalog_manager.cc:1347] Generated new cluster ID: 5ab07d8f317f435aa9b46dc79533c66e
I20260809 05:40:41.645397 31145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:41.659696 31145 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:41.660214 31145 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:41.675612 31145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03f6c323f90d474d8279a2f2d5692358: Generated new TSK 0
I20260809 05:40:41.676949 31145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:41.685281 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31005
W20260809 05:40:41.686494 31110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.97:33879
W20260809 05:40:41.687258 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31005: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:41.687413 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31005
2026-08-09T05:40:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:41 dist-test-slave-3kk6 krb5kdc[31172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:41 dist-test-slave-3kk6 krb5kdc[31172](info): set up 2 sockets
Aug 09 05:40:41 dist-test-slave-3kk6 krb5kdc[31172](info): commencing operation
krb5kdc: starting...
W20260809 05:40:43.071934 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:43 dist-test-slave-3kk6 krb5kdc[31172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:43.432284 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43027
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:43.792939 31190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:43.793249 31190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:43.793448 31190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:43.800772 31190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:43.800887 31190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:43.801020 31190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:43.801128 31190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:43.801218 31190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:43.811106 31190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:43027
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:43.812415 31190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:43.813787 31190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:43.825867 31196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:43.825776 31197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:43.825809 31200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:43.826978 31190 server_base.cc:1034] running on GCE node
I20260809 05:40:43.827785 31190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:43.829090 31190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:43.830375 31190 hybrid_clock.cc:648] HybridClock initialized: now 1786254043830318 us; error 58 us; skew 500 ppm
I20260809 05:40:43.830832 31190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:43 dist-test-slave-3kk6 krb5kdc[31172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254043, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:43.834033 31190 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:43.835624 31190 webserver.cc:460] Webserver started at http://127.30.137.254:36963/ using document root <none> and password file <none>
I20260809 05:40:43.836330 31190 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:43.836494 31190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:43.836807 31190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:43.839949 31190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "8c62d56908ae4ce98954f77cf0a89b44"
format_stamp: "Formatted at 2026-08-09 05:40:43 on dist-test-slave-3kk6"
server_key: "e6c20fe1131376a6397ede2d2d050125"
server_key_iv: "b5271402e1413733b4557a95e48702c4"
server_key_version: "encryptionkey@0"
I20260809 05:40:43.840778 31190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "8c62d56908ae4ce98954f77cf0a89b44"
format_stamp: "Formatted at 2026-08-09 05:40:43 on dist-test-slave-3kk6"
server_key: "e6c20fe1131376a6397ede2d2d050125"
server_key_iv: "b5271402e1413733b4557a95e48702c4"
server_key_version: "encryptionkey@0"
I20260809 05:40:43.846196 31190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 05:40:43.849807 31208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:43.851209 31190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:43.851400 31190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "8c62d56908ae4ce98954f77cf0a89b44"
format_stamp: "Formatted at 2026-08-09 05:40:43 on dist-test-slave-3kk6"
server_key: "e6c20fe1131376a6397ede2d2d050125"
server_key_iv: "b5271402e1413733b4557a95e48702c4"
server_key_version: "encryptionkey@0"
I20260809 05:40:43.851611 31190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:43.865458 31190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:43.869668 31190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:43.870007 31190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:43.890117 31190 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:43027
I20260809 05:40:43.890136 31286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:43027 every 8 connection(s)
I20260809 05:40:43.891819 31190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:43.895659 31287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:43.898957 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31190
I20260809 05:40:43.899163 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:43.899772 31271 external_mini_cluster.cc:1428] Setting key cce825cb39395c8c1354f407072f2b0f
I20260809 05:40:43.908754 31287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44: Bootstrap starting.
I20260809 05:40:43.912787 31287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:43.914196 31287 log.cc:824] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:40:43 dist-test-slave-3kk6 krb5kdc[31172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:43.917590 31287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44: No bootstrap required, opened a new log
I20260809 05:40:43.924950 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:43.901774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44119 (local address 127.30.137.254:43027)
0809 05:40:43.902631 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:43.902643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:43.902742 (+ 99us) server_negotiation.cc:407] Connection header received
0809 05:40:43.904498 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:43.904521 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:43.904942 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:43.905303 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:43.907234 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:43.908594 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:43.910405 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:43.911166 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:43.911311 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:43.916933 (+ 5622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:43.916940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:43.916953 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:43.916990 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:43.920357 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:43.921271 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:43.921277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:43.921282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:43.921461 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:43.921990 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:43.921993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:43.921995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:43.922569 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:43.922882 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:43.923453 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 05:40:43.923881 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":431,"threads_started":1}
I20260809 05:40:43.926704 31287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c62d56908ae4ce98954f77cf0a89b44" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43027 } }
I20260809 05:40:43.927115 31287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:43.927250 31287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c62d56908ae4ce98954f77cf0a89b44, State: Initialized, Role: FOLLOWER
I20260809 05:40:43.927884 31287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c62d56908ae4ce98954f77cf0a89b44" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43027 } }
I20260809 05:40:43.928130 31287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:43.928303 31287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:43.928506 31287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:43.929203 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:43.930966 31287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c62d56908ae4ce98954f77cf0a89b44" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43027 } }
I20260809 05:40:43.931525 31287 leader_election.cc:302] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c62d56908ae4ce98954f77cf0a89b44; no voters:
I20260809 05:40:43.932205 31287 leader_election.cc:288] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:43.932317 31299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:43.933712 31299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [term 1 LEADER]: Becoming Leader. State: Replica: 8c62d56908ae4ce98954f77cf0a89b44, State: Running, Role: LEADER
I20260809 05:40:43.934369 31299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c62d56908ae4ce98954f77cf0a89b44" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43027 } }
I20260809 05:40:43.934723 31287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:43.937522 31304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c62d56908ae4ce98954f77cf0a89b44. Latest consensus state: current_term: 1 leader_uuid: "8c62d56908ae4ce98954f77cf0a89b44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c62d56908ae4ce98954f77cf0a89b44" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43027 } } }
I20260809 05:40:43.938071 31304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:43.938218 31301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c62d56908ae4ce98954f77cf0a89b44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c62d56908ae4ce98954f77cf0a89b44" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 43027 } } }
I20260809 05:40:43.938592 31301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:43.948994 31312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:43.956861 31312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:43.973475 31312 catalog_manager.cc:1347] Generated new cluster ID: 299332bc9850422d95ccf93edc0a3826
I20260809 05:40:43.973598 31312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:43.975102 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:43.941404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44131 (local address 127.30.137.254:43027)
0809 05:40:43.941621 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:43.941630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:43.950868 (+ 9238us) server_negotiation.cc:407] Connection header received
0809 05:40:43.951134 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:43.951139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:43.951222 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:43.951347 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:43.953658 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:43.955021 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:43.957209 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:43.957962 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:43.958082 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:43.961959 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:43.961966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:43.961972 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:43.962022 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:43.966223 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:43.970384 (+ 4161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:43.970393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:43.970400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:43.970602 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:43.971346 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:43.971355 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:43.971361 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:43.972022 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:43.972397 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:43.974233 (+ 1836us) server_negotiation.cc:299] Negotiation successful
0809 05:40:43.974511 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 05:40:43.991189 31312 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:43.992453 31312 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:44.001492 31312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c62d56908ae4ce98954f77cf0a89b44: Generated new TSK 0
I20260809 05:40:44.002389 31312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:44.008555 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31190
I20260809 05:40:44.010128 31281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:44.105034 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31190
W20260809 05:40:44.127632 31168 connection.cc:535] client connection to 127.30.137.254:43027 recv error: Network error: failed to read from TLS socket (remote: 127.30.137.254:43027): Connection reset by peer (error 104)
2026-08-09T05:40:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:44 dist-test-slave-3kk6 krb5kdc[31342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:44 dist-test-slave-3kk6 krb5kdc[31342](info): set up 2 sockets
Aug 09 05:40:44 dist-test-slave-3kk6 krb5kdc[31342](info): commencing operation
krb5kdc: starting...
W20260809 05:40:45.450369 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:45 dist-test-slave-3kk6 krb5kdc[31342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:45.823311 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35841
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:38211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:46.198932 31362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:46.199268 31362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:46.199476 31362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:46.207060 31362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:46.207201 31362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:46.207327 31362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:46.207417 31362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:46.207504 31362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:46.218122 31362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:38211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:46.219729 31362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:46.221144 31362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:46.234158 31369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:46.234088 31372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:46.234036 31370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:46.235481 31362 server_base.cc:1034] running on GCE node
I20260809 05:40:46.236318 31362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:46.237982 31362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:46.239223 31362 hybrid_clock.cc:648] HybridClock initialized: now 1786254046239170 us; error 50 us; skew 500 ppm
I20260809 05:40:46.239558 31362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:46 dist-test-slave-3kk6 krb5kdc[31342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254046, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:46.247893 31362 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:46.249676 31362 webserver.cc:460] Webserver started at http://127.30.137.254:32843/ using document root <none> and password file <none>
I20260809 05:40:46.250483 31362 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:46.250691 31362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:46.251047 31362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:46.254300 31362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "ff3ad98718e341fd94e1df70f460a88c"
format_stamp: "Formatted at 2026-08-09 05:40:46 on dist-test-slave-3kk6"
server_key: "7b4a1ae8ed71cb02ba148bd45b14f265"
server_key_iv: "39352c7631a59a9253b567fa582d379f"
server_key_version: "encryptionkey@0"
I20260809 05:40:46.255177 31362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "ff3ad98718e341fd94e1df70f460a88c"
format_stamp: "Formatted at 2026-08-09 05:40:46 on dist-test-slave-3kk6"
server_key: "7b4a1ae8ed71cb02ba148bd45b14f265"
server_key_iv: "39352c7631a59a9253b567fa582d379f"
server_key_version: "encryptionkey@0"
I20260809 05:40:46.261212 31362 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260809 05:40:46.265269 31381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:46.267009 31362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 05:40:46.267246 31362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "ff3ad98718e341fd94e1df70f460a88c"
format_stamp: "Formatted at 2026-08-09 05:40:46 on dist-test-slave-3kk6"
server_key: "7b4a1ae8ed71cb02ba148bd45b14f265"
server_key_iv: "39352c7631a59a9253b567fa582d379f"
server_key_version: "encryptionkey@0"
I20260809 05:40:46.267508 31362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:46.303161 31362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:46.308516 31362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:46.308885 31362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:46.330379 31362 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35841
I20260809 05:40:46.330411 31467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35841 every 8 connection(s)
I20260809 05:40:46.332137 31362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:46.336302 31468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:46.342049 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31362
I20260809 05:40:46.342284 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:46.342886 31271 external_mini_cluster.cc:1428] Setting key 516030c2c75be128903ea1fe713ed84f
I20260809 05:40:46.348335 31468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c: Bootstrap starting.
I20260809 05:40:46.352406 31468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:46.353693 31468 log.cc:824] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:46.356909 31468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c: No bootstrap required, opened a new log
I20260809 05:40:46.357851 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:46.345049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41737 (local address 127.30.137.254:35841)
0809 05:40:46.346026 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:46.346039 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:46.346095 (+ 56us) server_negotiation.cc:407] Connection header received
0809 05:40:46.347953 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:46.347978 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:46.348411 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:46.348830 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:46.349570 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:46.351065 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:46.353059 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:46.353843 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:46.354072 (+ 229us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:46.355484 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:46.355491 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:46.355506 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:46.356077 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:46.356152 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:46.356183 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:46.356358 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:46.356649 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 05:40:46.356834 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":439,"threads_started":1}
I20260809 05:40:46.362020 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:46.365760 31468 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3ad98718e341fd94e1df70f460a88c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35841 } }
I20260809 05:40:46.366202 31468 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:46.366319 31468 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff3ad98718e341fd94e1df70f460a88c, State: Initialized, Role: FOLLOWER
I20260809 05:40:46.367105 31468 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3ad98718e341fd94e1df70f460a88c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35841 } }
I20260809 05:40:46.367437 31468 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:46.367602 31468 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:46.367895 31468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:46.371070 31468 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3ad98718e341fd94e1df70f460a88c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35841 } }
I20260809 05:40:46.371730 31468 leader_election.cc:302] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff3ad98718e341fd94e1df70f460a88c; no voters:
I20260809 05:40:46.372552 31468 leader_election.cc:288] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:46.372922 31486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:46.376864 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:46.366741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60645 (local address 127.30.137.254:35841)
0809 05:40:46.366960 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:46.366968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:46.367673 (+ 705us) server_negotiation.cc:407] Connection header received
0809 05:40:46.367808 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:46.367814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:46.367921 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:46.368039 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:46.368926 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:46.370138 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:46.371705 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:46.372617 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:46.373801 (+ 1184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:46.375311 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:46.375321 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:46.375328 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:46.375364 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:46.375456 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:46.375487 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:46.376143 (+ 656us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:46.376264 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 05:40:46.376340 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 05:40:46.378605 31468 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:46.381006 31486 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [term 1 LEADER]: Becoming Leader. State: Replica: ff3ad98718e341fd94e1df70f460a88c, State: Running, Role: LEADER
I20260809 05:40:46.381656 31486 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3ad98718e341fd94e1df70f460a88c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35841 } }
I20260809 05:40:46.384513 31488 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff3ad98718e341fd94e1df70f460a88c. Latest consensus state: current_term: 1 leader_uuid: "ff3ad98718e341fd94e1df70f460a88c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3ad98718e341fd94e1df70f460a88c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35841 } } }
I20260809 05:40:46.384969 31488 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:46.385608 31487 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff3ad98718e341fd94e1df70f460a88c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3ad98718e341fd94e1df70f460a88c" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35841 } } }
I20260809 05:40:46.385906 31487 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:46.387233 31506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:46.392454 31506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:46.402103 31506 catalog_manager.cc:1347] Generated new cluster ID: b52ebb2e05df40f0a12cf522c887533b
I20260809 05:40:46.402246 31506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:46.432785 31506 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:46.434772 31506 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:46.455596 31506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff3ad98718e341fd94e1df70f460a88c: Generated new TSK 0
I20260809 05:40:46.456547 31506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:46.469587 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31362
W20260809 05:40:46.471069 31460 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41737
W20260809 05:40:46.471963 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31362: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:46.472173 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31362
W20260809 05:40:46.503891 31340 connection.cc:535] client connection to 127.30.137.254:35841 recv error: Network error: failed to read from TLS socket (remote: 127.30.137.254:35841): Connection reset by peer (error 104)
2026-08-09T05:40:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:46 dist-test-slave-3kk6 krb5kdc[31522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:46 dist-test-slave-3kk6 krb5kdc[31522](info): set up 2 sockets
Aug 09 05:40:46 dist-test-slave-3kk6 krb5kdc[31522](info): commencing operation
krb5kdc: starting...
W20260809 05:40:47.843719 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:48 dist-test-slave-3kk6 krb5kdc[31522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:48.214251 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:37541
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:37541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:48.590492 31545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:48.590895 31545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:48.591154 31545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:48.599368 31545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:48.599505 31545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:48.599637 31545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:48.599761 31545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:48.599844 31545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:48.610148 31545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:37541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:37541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:48.611586 31545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:48.612932 31545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:48.625867 31554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:48.625845 31555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:48.625998 31557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:48.626773 31545 server_base.cc:1034] running on GCE node
I20260809 05:40:48.627611 31545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:48.629173 31545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:48.630455 31545 hybrid_clock.cc:648] HybridClock initialized: now 1786254048630412 us; error 40 us; skew 500 ppm
I20260809 05:40:48.630945 31545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:48 dist-test-slave-3kk6 krb5kdc[31522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254048, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:48.634334 31545 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:48.636118 31545 webserver.cc:460] Webserver started at http://127.30.137.254:44071/ using document root <none> and password file <none>
I20260809 05:40:48.636869 31545 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:48.637025 31545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:48.637362 31545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:48.640697 31545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "6ead80c310c8457d8e9ec7850ee4b435"
format_stamp: "Formatted at 2026-08-09 05:40:48 on dist-test-slave-3kk6"
server_key: "d977997fde766534cee3b054e7174f85"
server_key_iv: "370f77804b56a08951f4d7160fbbec90"
server_key_version: "encryptionkey@0"
I20260809 05:40:48.641539 31545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "6ead80c310c8457d8e9ec7850ee4b435"
format_stamp: "Formatted at 2026-08-09 05:40:48 on dist-test-slave-3kk6"
server_key: "d977997fde766534cee3b054e7174f85"
server_key_iv: "370f77804b56a08951f4d7160fbbec90"
server_key_version: "encryptionkey@0"
I20260809 05:40:48.647262 31545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 05:40:48.650991 31565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:48.652429 31545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:40:48.652585 31545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "6ead80c310c8457d8e9ec7850ee4b435"
format_stamp: "Formatted at 2026-08-09 05:40:48 on dist-test-slave-3kk6"
server_key: "d977997fde766534cee3b054e7174f85"
server_key_iv: "370f77804b56a08951f4d7160fbbec90"
server_key_version: "encryptionkey@0"
I20260809 05:40:48.652784 31545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:48.658860 31545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:48.663666 31545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:48.663955 31545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:48.684425 31545 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:37541
I20260809 05:40:48.684450 31647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:37541 every 8 connection(s)
I20260809 05:40:48.686038 31545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:48.690033 31648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:48.692345 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31545
I20260809 05:40:48.692548 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:48.693011 31271 external_mini_cluster.cc:1428] Setting key f35db355f45c4f1ee4c99a7ecd3d65af
I20260809 05:40:48.701273 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:48.694944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53343 (local address 127.30.137.254:37541)
0809 05:40:48.695776 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:48.695788 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:48.695831 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:40:48.697557 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:48.697582 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:48.697956 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:48.698390 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:48.699554 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:48.699563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:48.699576 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:48.699591 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:48.699652 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:48.699672 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:48.699954 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:48.700105 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 05:40:48.700303 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":370,"threads_started":1}
I20260809 05:40:48.702611 31648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435: Bootstrap starting.
I20260809 05:40:48.705216 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:48.706085 31648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:48.707194 31648 log.cc:824] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:48.710372 31648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435: No bootstrap required, opened a new log
I20260809 05:40:48.712787 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:48.709947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48911 (local address 127.30.137.254:37541)
0809 05:40:48.710142 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:48.710147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:48.710590 (+ 443us) server_negotiation.cc:407] Connection header received
0809 05:40:48.710791 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:48.710796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:48.710862 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:48.711003 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:48.712004 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:48.712013 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:48.712019 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:48.712038 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:48.712087 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:48.712112 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:48.712243 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:48.712395 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 05:40:48.712464 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 05:40:48.718957 31648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ead80c310c8457d8e9ec7850ee4b435" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37541 } }
I20260809 05:40:48.719395 31648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:48.719529 31648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ead80c310c8457d8e9ec7850ee4b435, State: Initialized, Role: FOLLOWER
I20260809 05:40:48.720140 31648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ead80c310c8457d8e9ec7850ee4b435" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37541 } }
I20260809 05:40:48.720393 31648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:48.720562 31648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:48.720745 31648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:48.723251 31648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ead80c310c8457d8e9ec7850ee4b435" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37541 } }
I20260809 05:40:48.723791 31648 leader_election.cc:302] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ead80c310c8457d8e9ec7850ee4b435; no voters:
I20260809 05:40:48.724525 31648 leader_election.cc:288] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:48.724663 31667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:48.725677 31667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [term 1 LEADER]: Becoming Leader. State: Replica: 6ead80c310c8457d8e9ec7850ee4b435, State: Running, Role: LEADER
I20260809 05:40:48.726305 31667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ead80c310c8457d8e9ec7850ee4b435" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37541 } }
I20260809 05:40:48.727306 31648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:48.729889 31669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ead80c310c8457d8e9ec7850ee4b435. Latest consensus state: current_term: 1 leader_uuid: "6ead80c310c8457d8e9ec7850ee4b435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ead80c310c8457d8e9ec7850ee4b435" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37541 } } }
I20260809 05:40:48.730283 31668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ead80c310c8457d8e9ec7850ee4b435" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ead80c310c8457d8e9ec7850ee4b435" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37541 } } }
I20260809 05:40:48.730624 31668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:48.730619 31669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435 [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:48.731819 31677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:48.738726 31677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:48.748486 31677 catalog_manager.cc:1347] Generated new cluster ID: 3a2a78bb06414baa9c0548d475d20234
I20260809 05:40:48.748634 31677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:48.763325 31677 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:48.763898 31677 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:48.775354 31677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ead80c310c8457d8e9ec7850ee4b435: Generated new TSK 0
I20260809 05:40:48.776547 31677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:48.790941 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31545
W20260809 05:40:48.792261 31638 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53343
W20260809 05:40:48.793069 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31545: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:48.793260 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31545
2026-08-09T05:40:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:48 dist-test-slave-3kk6 krb5kdc[31699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:48 dist-test-slave-3kk6 krb5kdc[31699](info): set up 2 sockets
Aug 09 05:40:48 dist-test-slave-3kk6 krb5kdc[31699](info): commencing operation
krb5kdc: starting...
W20260809 05:40:50.147338 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:50 dist-test-slave-3kk6 krb5kdc[31699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:50.514122 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34837
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:34837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:50.875556 31716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:50.875876 31716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:50.876058 31716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:50.883387 31716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:50.883543 31716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:50.883664 31716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:50.883771 31716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:50.883862 31716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:50.893934 31716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:34837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:34837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:50.895412 31716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:50.896793 31716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:50.909435 31723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:50.909466 31725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:50.909520 31728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:50.910336 31716 server_base.cc:1034] running on GCE node
I20260809 05:40:50.911206 31716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:50.912688 31716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:50.913934 31716 hybrid_clock.cc:648] HybridClock initialized: now 1786254050913876 us; error 54 us; skew 500 ppm
I20260809 05:40:50.914318 31716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:50 dist-test-slave-3kk6 krb5kdc[31699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254050, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:50.917565 31716 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:50.919215 31716 webserver.cc:460] Webserver started at http://127.30.137.254:40821/ using document root <none> and password file <none>
I20260809 05:40:50.919937 31716 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:50.920089 31716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:50.920398 31716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:50.923612 31716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "bdc3b166f36840faa980b65f34f99fbe"
format_stamp: "Formatted at 2026-08-09 05:40:50 on dist-test-slave-3kk6"
server_key: "785695711f2643ffd795fe2767faeb85"
server_key_iv: "902c3a7ec20104ab699dd39d06d80eec"
server_key_version: "encryptionkey@0"
I20260809 05:40:50.924449 31716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "bdc3b166f36840faa980b65f34f99fbe"
format_stamp: "Formatted at 2026-08-09 05:40:50 on dist-test-slave-3kk6"
server_key: "785695711f2643ffd795fe2767faeb85"
server_key_iv: "902c3a7ec20104ab699dd39d06d80eec"
server_key_version: "encryptionkey@0"
I20260809 05:40:50.929911 31716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 05:40:50.933611 31737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:50.935075 31716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:50.935271 31716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "bdc3b166f36840faa980b65f34f99fbe"
format_stamp: "Formatted at 2026-08-09 05:40:50 on dist-test-slave-3kk6"
server_key: "785695711f2643ffd795fe2767faeb85"
server_key_iv: "902c3a7ec20104ab699dd39d06d80eec"
server_key_version: "encryptionkey@0"
I20260809 05:40:50.935488 31716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:50.952385 31716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:50.957108 31716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:50.957429 31716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:50.977166 31716 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:34837
I20260809 05:40:50.977190 31819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:34837 every 8 connection(s)
I20260809 05:40:50.978811 31716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:50.980680 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31716
I20260809 05:40:50.980902 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:50.981388 31271 external_mini_cluster.cc:1428] Setting key 527cbf5b350c69d5fdbfd40d4dd0c1af
I20260809 05:40:50.983371 31821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 05:40:50 dist-test-slave-3kk6 krb5kdc[31699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:50.997541 31821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe: Bootstrap starting.
I20260809 05:40:51.001807 31821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:51.003278 31821 log.cc:824] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:51.007030 31821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe: No bootstrap required, opened a new log
I20260809 05:40:51.008036 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:50.983454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35895 (local address 127.30.137.254:34837)
0809 05:40:50.984337 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:50.984355 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:50.984412 (+ 57us) server_negotiation.cc:407] Connection header received
0809 05:40:50.986643 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:50.986701 (+ 58us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:50.987257 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:50.987731 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:50.989139 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:50.991280 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:50.992664 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:50.993725 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:50.993941 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:50.998693 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:50.998707 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:50.998728 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:50.998777 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:51.003057 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:51.003951 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:51.003965 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:51.003973 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:51.004165 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:51.004633 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:51.004636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:51.004638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:51.005381 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:51.005671 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:51.006119 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 05:40:51.006569 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":357,"threads_started":1}
I20260809 05:40:51.011950 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:51.015772 31821 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc3b166f36840faa980b65f34f99fbe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34837 } }
I20260809 05:40:51.016351 31821 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:51.016551 31821 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc3b166f36840faa980b65f34f99fbe, State: Initialized, Role: FOLLOWER
I20260809 05:40:51.017191 31821 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc3b166f36840faa980b65f34f99fbe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34837 } }
I20260809 05:40:51.017457 31821 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:51.017627 31821 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:51.017820 31821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:51.020218 31821 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc3b166f36840faa980b65f34f99fbe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34837 } }
I20260809 05:40:51.020756 31821 leader_election.cc:302] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdc3b166f36840faa980b65f34f99fbe; no voters:
I20260809 05:40:51.021448 31821 leader_election.cc:288] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:51.021598 31835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:51.022291 31835 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [term 1 LEADER]: Becoming Leader. State: Replica: bdc3b166f36840faa980b65f34f99fbe, State: Running, Role: LEADER
I20260809 05:40:51.023028 31835 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc3b166f36840faa980b65f34f99fbe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34837 } }
I20260809 05:40:51.024452 31821 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:51.026262 31838 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdc3b166f36840faa980b65f34f99fbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc3b166f36840faa980b65f34f99fbe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34837 } } }
I20260809 05:40:51.026718 31838 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:51.027277 31836 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdc3b166f36840faa980b65f34f99fbe. Latest consensus state: current_term: 1 leader_uuid: "bdc3b166f36840faa980b65f34f99fbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc3b166f36840faa980b65f34f99fbe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 34837 } } }
I20260809 05:40:51.027623 31836 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:51.036484 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:51.016395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55337 (local address 127.30.137.254:34837)
0809 05:40:51.016610 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:51.016620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:51.017068 (+ 448us) server_negotiation.cc:407] Connection header received
0809 05:40:51.017284 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:51.017292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:51.017381 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:51.017492 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:51.019159 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:51.019915 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:51.021369 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:51.022241 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:51.022442 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:51.025834 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:51.025842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:51.025849 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:51.025898 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:51.030810 (+ 4912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:51.032045 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:51.032055 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:51.032061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:51.032370 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:51.032978 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:51.032986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:51.032992 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:51.033523 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:51.033917 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:51.035583 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0809 05:40:51.035928 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 05:40:51.039222 31845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:51.046718 31845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:51.057406 31845 catalog_manager.cc:1347] Generated new cluster ID: fe98e3d5424643458c98d1d2d7286cef
I20260809 05:40:51.057569 31845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:51.079614 31845 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:51.081177 31845 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:51.090569 31845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdc3b166f36840faa980b65f34f99fbe: Generated new TSK 0
I20260809 05:40:51.091500 31845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:51.099242 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31716
I20260809 05:40:51.101130 31815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:40:51.196506 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31716
2026-08-09T05:40:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:51 dist-test-slave-3kk6 krb5kdc[31867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:51 dist-test-slave-3kk6 krb5kdc[31867](info): set up 2 sockets
Aug 09 05:40:51 dist-test-slave-3kk6 krb5kdc[31867](info): commencing operation
krb5kdc: starting...
W20260809 05:40:52.579897 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.000s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:52 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:52.938258 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:41403
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:46099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:53.297591 31892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:53.297976 31892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:53.298146 31892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:53.305701 31892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:53.305898 31892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:53.306010 31892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:53.306125 31892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:53.306205 31892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:53.316500 31892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:46099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:53.317900 31892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:53.319288 31892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:53.331290 31905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:53.331247 31901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:53.331426 31903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:53.332902 31892 server_base.cc:1034] running on GCE node
I20260809 05:40:53.333753 31892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:53.335403 31892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:53.336644 31892 hybrid_clock.cc:648] HybridClock initialized: now 1786254053336595 us; error 33 us; skew 500 ppm
I20260809 05:40:53.337073 31892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:53 dist-test-slave-3kk6 krb5kdc[31867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254053, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:53.340588 31892 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:53.342223 31892 webserver.cc:460] Webserver started at http://127.30.137.254:39239/ using document root <none> and password file <none>
I20260809 05:40:53.343061 31892 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:53.343204 31892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:53.343546 31892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:53.346771 31892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "4b258172b9ef4b74bf567947699005eb"
format_stamp: "Formatted at 2026-08-09 05:40:53 on dist-test-slave-3kk6"
server_key: "4af54d45d38a365cc7c9565b1fbf94b1"
server_key_iv: "d6dd0e6d649bc4d56c526090ce3fa2e8"
server_key_version: "encryptionkey@0"
I20260809 05:40:53.347610 31892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "4b258172b9ef4b74bf567947699005eb"
format_stamp: "Formatted at 2026-08-09 05:40:53 on dist-test-slave-3kk6"
server_key: "4af54d45d38a365cc7c9565b1fbf94b1"
server_key_iv: "d6dd0e6d649bc4d56c526090ce3fa2e8"
server_key_version: "encryptionkey@0"
I20260809 05:40:53.353138 31892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:53.356925 31916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:53.358350 31892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:40:53.358549 31892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "4b258172b9ef4b74bf567947699005eb"
format_stamp: "Formatted at 2026-08-09 05:40:53 on dist-test-slave-3kk6"
server_key: "4af54d45d38a365cc7c9565b1fbf94b1"
server_key_iv: "d6dd0e6d649bc4d56c526090ce3fa2e8"
server_key_version: "encryptionkey@0"
I20260809 05:40:53.358852 31892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:53.398147 31892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:53.403123 31892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:53.403488 31892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:53.423447 31892 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:41403
I20260809 05:40:53.423477 32000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:41403 every 8 connection(s)
I20260809 05:40:53.424892 31892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:53.425230 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 31892
I20260809 05:40:53.425424 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:53.426004 31271 external_mini_cluster.cc:1428] Setting key 60df676ff9a01c76ede37c713595be9b
I20260809 05:40:53.429670 32001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:53.439919 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:53.428043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46211 (local address 127.30.137.254:41403)
0809 05:40:53.428874 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:53.428887 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:53.428930 (+ 43us) server_negotiation.cc:407] Connection header received
0809 05:40:53.430801 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:53.430823 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:53.431209 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:53.431599 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:53.432251 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:53.433619 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:53.435140 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:53.435879 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:53.436171 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:53.437429 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:53.437440 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:53.437455 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:53.438038 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:53.438123 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:53.438148 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:53.438320 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:53.438578 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 05:40:53.438904 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":375,"threads_started":1}
I20260809 05:40:53.441358 32001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb: Bootstrap starting.
I20260809 05:40:53.443995 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:53.444983 32001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:53.446036 32001 log.cc:824] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:53.449334 32001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb: No bootstrap required, opened a new log
I20260809 05:40:53.456094 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:53.448829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42361 (local address 127.30.137.254:41403)
0809 05:40:53.449063 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:53.449070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:53.449414 (+ 344us) server_negotiation.cc:407] Connection header received
0809 05:40:53.449658 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:53.449662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:53.449722 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:53.449860 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:53.450407 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:53.451235 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:53.453317 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:53.453774 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:53.453875 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:53.455222 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:53.455227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:53.455230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:53.455255 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:53.455294 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:53.455312 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:53.455388 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:53.455623 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 05:40:53.455704 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 05:40:53.459015 32001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b258172b9ef4b74bf567947699005eb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41403 } }
I20260809 05:40:53.459478 32001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:53.459601 32001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b258172b9ef4b74bf567947699005eb, State: Initialized, Role: FOLLOWER
I20260809 05:40:53.460204 32001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b258172b9ef4b74bf567947699005eb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41403 } }
I20260809 05:40:53.460482 32001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:53.460632 32001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:53.460825 32001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:53.463505 32001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b258172b9ef4b74bf567947699005eb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41403 } }
I20260809 05:40:53.464061 32001 leader_election.cc:302] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b258172b9ef4b74bf567947699005eb; no voters:
I20260809 05:40:53.464663 32001 leader_election.cc:288] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:53.464852 32016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:53.465969 32016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [term 1 LEADER]: Becoming Leader. State: Replica: 4b258172b9ef4b74bf567947699005eb, State: Running, Role: LEADER
I20260809 05:40:53.466732 32016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b258172b9ef4b74bf567947699005eb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41403 } }
I20260809 05:40:53.467053 32001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:53.469960 32018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b258172b9ef4b74bf567947699005eb. Latest consensus state: current_term: 1 leader_uuid: "4b258172b9ef4b74bf567947699005eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b258172b9ef4b74bf567947699005eb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41403 } } }
I20260809 05:40:53.469945 32017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b258172b9ef4b74bf567947699005eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b258172b9ef4b74bf567947699005eb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 41403 } } }
I20260809 05:40:53.470439 32018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:53.470496 32017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb [sys.catalog]: This master's current role is: LEADER
W20260809 05:40:53.482298 32038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:40:53.482455 32038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:40:53.482640 32025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:53.484859 32025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:53.494215 32025 catalog_manager.cc:1347] Generated new cluster ID: ce3d74373af54868b67bee25995b3507
I20260809 05:40:53.494379 32025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:53.511305 32025 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:53.512650 32025 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:53.524580 32025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b258172b9ef4b74bf567947699005eb: Generated new TSK 0
I20260809 05:40:53.525709 32025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:53.536557 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31892
W20260809 05:40:53.537767 31994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46211
W20260809 05:40:53.538542 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 31892: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:53.538743 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 31892
2026-08-09T05:40:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:53 dist-test-slave-3kk6 krb5kdc[32043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:53 dist-test-slave-3kk6 krb5kdc[32043](info): set up 2 sockets
Aug 09 05:40:53 dist-test-slave-3kk6 krb5kdc[32043](info): commencing operation
krb5kdc: starting...
W20260809 05:40:54.905220 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.005s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:55 dist-test-slave-3kk6 krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:55.266486 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39651
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:39651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:55.623103 32063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:55.623453 32063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:55.623631 32063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:55.631125 32063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:55.631299 32063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:55.631398 32063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:55.631516 32063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:55.631595 32063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:55.641618 32063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:39651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:39651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:55.643052 32063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:55.644439 32063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:55.657011 32071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:55.656934 32074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:55.657037 32076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:55.657936 32063 server_base.cc:1034] running on GCE node
I20260809 05:40:55.658792 32063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:55.660239 32063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:55.661504 32063 hybrid_clock.cc:648] HybridClock initialized: now 1786254055661465 us; error 37 us; skew 500 ppm
I20260809 05:40:55.661908 32063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:55 dist-test-slave-3kk6 krb5kdc[32043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254055, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:55.665552 32063 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:55.667402 32063 webserver.cc:460] Webserver started at http://127.30.137.254:46245/ using document root <none> and password file <none>
I20260809 05:40:55.668154 32063 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:55.668315 32063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:55.668655 32063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:55.671967 32063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "b31be970607c449aba9d5e154052f2fe"
format_stamp: "Formatted at 2026-08-09 05:40:55 on dist-test-slave-3kk6"
server_key: "fef3af5e4bda2502c69e0759c36750e6"
server_key_iv: "99d0b6b51336b9d844be2b5d408c9e17"
server_key_version: "encryptionkey@0"
I20260809 05:40:55.672807 32063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "b31be970607c449aba9d5e154052f2fe"
format_stamp: "Formatted at 2026-08-09 05:40:55 on dist-test-slave-3kk6"
server_key: "fef3af5e4bda2502c69e0759c36750e6"
server_key_iv: "99d0b6b51336b9d844be2b5d408c9e17"
server_key_version: "encryptionkey@0"
I20260809 05:40:55.678335 32063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 05:40:55.682019 32087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:55.683396 32063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:40:55.683600 32063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "b31be970607c449aba9d5e154052f2fe"
format_stamp: "Formatted at 2026-08-09 05:40:55 on dist-test-slave-3kk6"
server_key: "fef3af5e4bda2502c69e0759c36750e6"
server_key_iv: "99d0b6b51336b9d844be2b5d408c9e17"
server_key_version: "encryptionkey@0"
I20260809 05:40:55.683835 32063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:55.689977 32063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:55.694494 32063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:55.694869 32063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:55.714537 32063 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:39651
I20260809 05:40:55.714637 32161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:39651 every 8 connection(s)
I20260809 05:40:55.716101 32063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:55.720213 32162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:55.721484 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32063
I20260809 05:40:55.721719 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:55.722360 31271 external_mini_cluster.cc:1428] Setting key d4d9857461f00f28ecb42d73e94d7acc
I20260809 05:40:55.731149 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:55.724540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43983 (local address 127.30.137.254:39651)
0809 05:40:55.725420 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:55.725435 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:55.725477 (+ 42us) server_negotiation.cc:407] Connection header received
0809 05:40:55.727531 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:55.727559 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:55.727916 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:55.728356 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:55.729453 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:55.729459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:55.729473 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:55.729488 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:55.729550 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:55.729573 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:55.729831 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:55.729969 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 05:40:55.730157 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":378,"threads_started":1}
I20260809 05:40:55.735428 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:55.735888 32162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe: Bootstrap starting.
I20260809 05:40:55.739355 32162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:55.740924 32162 log.cc:824] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:55.742915 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:55.740020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57033 (local address 127.30.137.254:39651)
0809 05:40:55.740188 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:55.740192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:55.740755 (+ 563us) server_negotiation.cc:407] Connection header received
0809 05:40:55.740962 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:55.740975 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:55.741023 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:55.741135 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:55.742180 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:55.742186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:55.742191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 05:40:55.742210 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:55.742257 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 05:40:55.742273 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:55.742397 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:55.742522 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 05:40:55.742588 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 05:40:55.744750 32162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe: No bootstrap required, opened a new log
I20260809 05:40:55.753938 32162 raft_consensus.cc:348] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31be970607c449aba9d5e154052f2fe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39651 } }
I20260809 05:40:55.754421 32162 raft_consensus.cc:374] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 05:40:55.754549 32162 raft_consensus.cc:729] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b31be970607c449aba9d5e154052f2fe, State: Initialized, Role: FOLLOWER
I20260809 05:40:55.755224 32162 consensus_queue.cc:260] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31be970607c449aba9d5e154052f2fe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39651 } }
I20260809 05:40:55.755510 32162 raft_consensus.cc:388] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 05:40:55.755681 32162 raft_consensus.cc:482] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 05:40:55.755880 32162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:55.758433 32162 raft_consensus.cc:504] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31be970607c449aba9d5e154052f2fe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39651 } }
I20260809 05:40:55.759029 32162 leader_election.cc:302] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b31be970607c449aba9d5e154052f2fe; no voters:
I20260809 05:40:55.759742 32162 leader_election.cc:288] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 05:40:55.759828 32176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:55.760823 32176 raft_consensus.cc:686] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [term 1 LEADER]: Becoming Leader. State: Replica: b31be970607c449aba9d5e154052f2fe, State: Running, Role: LEADER
I20260809 05:40:55.761564 32176 consensus_queue.cc:237] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31be970607c449aba9d5e154052f2fe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39651 } }
I20260809 05:40:55.762820 32162 sys_catalog.cc:564] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:55.764730 32179 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b31be970607c449aba9d5e154052f2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31be970607c449aba9d5e154052f2fe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39651 } } }
I20260809 05:40:55.765255 32179 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:55.765022 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b31be970607c449aba9d5e154052f2fe. Latest consensus state: current_term: 1 leader_uuid: "b31be970607c449aba9d5e154052f2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b31be970607c449aba9d5e154052f2fe" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 39651 } } }
I20260809 05:40:55.765410 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:55.767463 32183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:55.774288 32183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:55.784018 32183 catalog_manager.cc:1347] Generated new cluster ID: a106dde7073d475987212f554aeeb7d0
I20260809 05:40:55.784159 32183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:55.826113 32183 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:55.826725 32183 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:55.838498 32183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b31be970607c449aba9d5e154052f2fe: Generated new TSK 0
I20260809 05:40:55.839713 32183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 05:40:55.845254 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32063
W20260809 05:40:55.846576 32155 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43983
W20260809 05:40:55.847431 31271 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32063: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 05:40:55.847599 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32063
2026-08-09T05:40:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2307 ms)
[----------] 18 tests from AuthTokenIssuingTest (42216 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:40:55 dist-test-slave-3kk6 krb5kdc[32212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:40:55 dist-test-slave-3kk6 krb5kdc[32212](info): set up 2 sockets
Aug 09 05:40:55 dist-test-slave-3kk6 krb5kdc[32212](info): commencing operation
krb5kdc: starting...
W20260809 05:40:57.213380 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.001s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:40:57 dist-test-slave-3kk6 krb5kdc[32212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:40:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:40:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:40:57.580483 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35523
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35523,127.30.137.253:38235,127.30.137.252:36893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:40:57.949474 32235 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:40:57.949879 32235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:57.950016 32235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:57.950162 32235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:57.957782 32235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:57.957892 32235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:57.957995 32235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:57.958088 32235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:57.958207 32235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:57.968667 32235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35523,127.30.137.253:38235,127.30.137.252:36893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:35523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:57.970062 32235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:57.971494 32235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:57.983812 32245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:57.983891 32246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:57.983827 32249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:57.985025 32235 server_base.cc:1034] running on GCE node
I20260809 05:40:57.985821 32235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:57.987411 32235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:57.988636 32235 hybrid_clock.cc:648] HybridClock initialized: now 1786254057988570 us; error 40 us; skew 500 ppm
I20260809 05:40:57.988997 32235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:57 dist-test-slave-3kk6 krb5kdc[32212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:57.998332 32235 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:40:58.000159 32235 webserver.cc:460] Webserver started at http://127.30.137.254:40409/ using document root <none> and password file <none>
I20260809 05:40:58.000945 32235 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:58.001091 32235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:58.001446 32235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:58.004706 32235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "c454cd78df4141cc9fde76a38fb489fb"
format_stamp: "Formatted at 2026-08-09 05:40:57 on dist-test-slave-3kk6"
server_key: "2115ac7ace377602ccd3553273e8c945"
server_key_iv: "916c39549cc2b9dc7cc06ab0df41b0b1"
server_key_version: "encryptionkey@0"
I20260809 05:40:58.005574 32235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "c454cd78df4141cc9fde76a38fb489fb"
format_stamp: "Formatted at 2026-08-09 05:40:57 on dist-test-slave-3kk6"
server_key: "2115ac7ace377602ccd3553273e8c945"
server_key_iv: "916c39549cc2b9dc7cc06ab0df41b0b1"
server_key_version: "encryptionkey@0"
I20260809 05:40:58.011143 32235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:40:58.014938 32262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:58.016400 32235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:40:58.016619 32235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "c454cd78df4141cc9fde76a38fb489fb"
format_stamp: "Formatted at 2026-08-09 05:40:57 on dist-test-slave-3kk6"
server_key: "2115ac7ace377602ccd3553273e8c945"
server_key_iv: "916c39549cc2b9dc7cc06ab0df41b0b1"
server_key_version: "encryptionkey@0"
I20260809 05:40:58.016881 32235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:58.035259 32235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:58.039733 32235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:58.040058 32235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:58.060302 32235 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:35523
I20260809 05:40:58.060324 32329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:35523 every 8 connection(s)
I20260809 05:40:58.061841 32235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:40:58.065702 32330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:58.068722 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32235
I20260809 05:40:58.068969 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:40:58.069672 31271 external_mini_cluster.cc:1428] Setting key 0b3f8650e41d5c28e6f97f1859c2e36f
I20260809 05:40:58.072818 32330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:40:58 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:58.096650 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.077648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.30.137.254:35523)
0809 05:40:58.078450 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:58.078465 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:58.078520 (+ 55us) server_negotiation.cc:407] Connection header received
0809 05:40:58.079875 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:58.079881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:58.080129 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:58.080351 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:58.081699 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.082961 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.084367 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.084935 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.085168 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.088777 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:58.088793 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:58.088806 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:58.088835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:58.091864 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.092613 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.092628 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.092630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.092804 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.093275 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.093282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.093285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.093825 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:58.094052 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:58.094633 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 05:40:58.095056 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":373,"threads_started":1}
I20260809 05:40:58.096686 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.076281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35523 (local address 127.0.0.1:42842)
0809 05:40:58.077115 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.077156 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:58.079456 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:58.080377 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:58.080384 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:58.081125 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:58.081470 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.081487 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.083124 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.083132 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.084214 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.084226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.085182 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.085189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.085407 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.086141 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:58.086164 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:58.088542 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:58.092017 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.092029 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.092042 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.092474 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.092924 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.092932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.092935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.093166 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.093967 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:58.093981 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:58.094243 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 05:40:58.094525 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 05:40:58.095204 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":176,"thread_start_us":357,"threads_started":1}
I20260809 05:40:58.101292 32330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:40:58.103753 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.103121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47810)
0809 05:40:58.103360 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.103511 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260809 05:40:58.104261 32265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.137.253@KRBTEST.COM; defaulting to no policy
I20260809 05:40:58.106210 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.105727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47824)
0809 05:40:58.105923 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.106017 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 05:40:58.106570 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:38235: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab.
Entry for principal kudu/127.30.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab.
I20260809 05:40:58.150547 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:38235
--webserver_interface=127.30.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab
--principal=kudu/127.30.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35523,127.30.137.253:38235,127.30.137.252:36893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:40:58.156030 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } attempt: 1
I20260809 05:40:58.157346 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.156832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47826)
0809 05:40:58.157046 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.157190 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 05:40:58.158416 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.158046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47830)
0809 05:40:58.158212 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.158281 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 05:40:58.158675 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:38235: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
I20260809 05:40:58.227003 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } attempt: 2
I20260809 05:40:58.228469 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.227830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47834)
0809 05:40:58.228127 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.228280 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 05:40:58.229796 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.229338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47840)
0809 05:40:58.229519 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.229616 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 05:40:58.230055 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:38235: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
I20260809 05:40:58.321393 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } attempt: 3
I20260809 05:40:58.322788 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.322165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47842)
0809 05:40:58.322434 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.322559 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 05:40:58.323918 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.323504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47844)
0809 05:40:58.323660 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.323750 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 05:40:58.324131 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:38235: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
I20260809 05:40:58.467463 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } attempt: 4
I20260809 05:40:58.468885 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.468253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47846)
0809 05:40:58.468599 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.468713 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260809 05:40:58.470108 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.469661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47856)
0809 05:40:58.469840 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.469935 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 05:40:58.470368 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:38235: Network error: Client connection negotiation failed: client connection to 127.30.137.253:38235: connect: Connection refused (error 111)
W20260809 05:40:58.515100 32339 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:40:58.515439 32339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:58.515514 32339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:58.515595 32339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:58.523061 32339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:58.523173 32339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:58.523236 32339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:58.523288 32339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:58.523345 32339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:58.533351 32339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35523,127.30.137.253:38235,127.30.137.252:36893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab
--principal=kudu/127.30.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:58.534614 32339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:58.535957 32339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:58.548350 32347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:58.548431 32345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:58.548406 32352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:58.549304 32339 server_base.cc:1034] running on GCE node
I20260809 05:40:58.550136 32339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:58.551739 32339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:58.552982 32339 hybrid_clock.cc:648] HybridClock initialized: now 1786254058552912 us; error 38 us; skew 500 ppm
I20260809 05:40:58.553382 32339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:58 dist-test-slave-3kk6 krb5kdc[32212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:58.556757 32339 init.cc:377] Logged in from keytab as kudu/127.30.137.253@KRBTEST.COM (short username kudu)
I20260809 05:40:58.558350 32339 webserver.cc:460] Webserver started at http://127.30.137.253:35291/ using document root <none> and password file <none>
I20260809 05:40:58.559144 32339 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:58.559240 32339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:58.559540 32339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:58.562949 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data/instance:
uuid: "d8b504da920c442495dedf1bc687c22c"
format_stamp: "Formatted at 2026-08-09 05:40:58 on dist-test-slave-3kk6"
server_key: "d22ea4b7f890a50f5ff18d020bff1f75"
server_key_iv: "df2a197b3c05787d784be1f0e9a561fb"
server_key_version: "encryptionkey@0"
I20260809 05:40:58.563786 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal/instance:
uuid: "d8b504da920c442495dedf1bc687c22c"
format_stamp: "Formatted at 2026-08-09 05:40:58 on dist-test-slave-3kk6"
server_key: "d22ea4b7f890a50f5ff18d020bff1f75"
server_key_iv: "df2a197b3c05787d784be1f0e9a561fb"
server_key_version: "encryptionkey@0"
I20260809 05:40:58.569310 32339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 05:40:58.572971 32361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:58.574359 32339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:40:58.574504 32339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal
uuid: "d8b504da920c442495dedf1bc687c22c"
format_stamp: "Formatted at 2026-08-09 05:40:58 on dist-test-slave-3kk6"
server_key: "d22ea4b7f890a50f5ff18d020bff1f75"
server_key_iv: "df2a197b3c05787d784be1f0e9a561fb"
server_key_version: "encryptionkey@0"
I20260809 05:40:58.574836 32339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:58.591614 32339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:58.596174 32339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:58.596421 32339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:58.616477 32339 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.253:38235
I20260809 05:40:58.616496 32438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.253:38235 every 8 connection(s)
I20260809 05:40:58.617915 32339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
I20260809 05:40:58.618381 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32339
I20260809 05:40:58.618580 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-1/wal/instance
I20260809 05:40:58.619232 31271 external_mini_cluster.cc:1428] Setting key f8048e9dd2ba8f2575dba72821d5355f
I20260809 05:40:58.622056 32439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:58.628705 32439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:40:58 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:58.650077 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.631459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.30.137.254:35523)
0809 05:40:58.631983 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:58.631988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:58.633032 (+ 1044us) server_negotiation.cc:407] Connection header received
0809 05:40:58.635080 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:58.635085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:58.635190 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:58.635316 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:58.636944 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.637689 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.639371 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.639797 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.639939 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.643705 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:58.643710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:58.643715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:58.643768 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:58.646697 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.647536 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.647542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.647547 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.647690 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.648157 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.648168 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.648171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.648469 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:58.648646 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:58.649425 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 05:40:58.649675 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":304,"threads_started":1}
I20260809 05:40:58.651238 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.632063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35523 (local address 127.0.0.1:42856)
0809 05:40:58.632856 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.632899 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:58.634790 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:58.635562 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:58.635570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:58.636113 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:58.636702 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.636720 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.637873 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.637877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.639205 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.639225 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.640047 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.640054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.640301 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.641090 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:58.641119 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:58.643455 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:58.646866 (+ 3411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.646876 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.646895 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.647394 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.647817 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.647822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.647826 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.648025 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.648608 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:58.648617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:58.649026 (+ 409us) client_negotiation.cc:769] Sending connection context
0809 05:40:58.649324 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 05:40:58.649916 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":320,"threads_started":1}
WARNING: no policy specified for kudu/127.30.137.252@KRBTEST.COM; defaulting to no policy
I20260809 05:40:58.653159 32439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.252@KRBTEST.COM" created.
Aug 09 05:40:58 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:40:58.670888 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.654024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47868)
0809 05:40:58.654202 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.654229 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:58.654326 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:58.655313 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:58.655316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:58.655883 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:58.656031 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.656039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.658166 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.658170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.659096 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.659108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.660363 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.660367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.660551 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.661230 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:58.661250 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:58.663589 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:58.668242 (+ 4653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.668247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.668250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.668726 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.669179 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.669182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.669184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.669323 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.670085 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:58.670088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:58.670207 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 05:40:58.670331 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 05:40:58.670541 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":36}
I20260809 05:40:58.671417 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.654077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.30.137.253:38235)
0809 05:40:58.654855 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:58.654863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:58.654898 (+ 35us) server_negotiation.cc:407] Connection header received
0809 05:40:58.654950 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:58.654956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:58.655134 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:58.655265 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:58.656219 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.658004 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.659236 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.660202 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.660364 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.663842 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:58.663858 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:58.663879 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:58.663923 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:58.668057 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.668867 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.668880 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.668886 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.669041 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.669432 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.669439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.669445 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.669957 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:58.670189 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:58.670526 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 05:40:58.670858 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":504,"thread_start_us":362,"threads_started":1}
I20260809 05:40:58.674080 32439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:40:58.675645 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.675022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60346)
0809 05:40:58.675291 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.675437 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 05:40:58.675932 32363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:58.677584 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.677221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60350)
0809 05:40:58.677397 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.677478 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 05:40:58.677963 32439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab.
Entry for principal kudu/127.30.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab.
I20260809 05:40:58.701598 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:36893
--webserver_interface=127.30.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab
--principal=kudu/127.30.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:35523,127.30.137.253:38235,127.30.137.252:36893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:40:58.727337 32439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 1
I20260809 05:40:58.728709 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.728234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60362)
0809 05:40:58.728434 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.728554 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 05:40:58.729832 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.729459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60368)
0809 05:40:58.729642 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.729720 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 05:40:58.730043 32439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:58.769634 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } attempt: 5
Aug 09 05:40:58 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:40:58.783834 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.770628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47884 (local address 127.30.137.253:38235)
0809 05:40:58.770824 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:58.770829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:58.770925 (+ 96us) server_negotiation.cc:407] Connection header received
0809 05:40:58.771160 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:58.771165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:58.771257 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:58.771372 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:58.772233 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.772961 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.774190 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.774725 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.774861 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.778278 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:58.778287 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:58.778294 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:58.778336 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:58.781190 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.781853 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.781863 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.781865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.781987 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:58.782384 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:58.782390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:58.782394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:58.782734 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:58.782890 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:58.783281 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 05:40:58.783507 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 05:40:58.783857 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.770528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47884)
0809 05:40:58.770796 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.770821 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:58.771009 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:58.771430 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:58.771436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:58.771909 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:58.772067 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.772076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.773103 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.773111 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.774031 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:58.774043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:58.774888 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:58.774894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:58.775124 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:58.775856 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:58.775874 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:58.778039 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:58.781342 (+ 3303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.781347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.781351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.781712 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.782104 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:58.782109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:58.782113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:58.782272 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:58.782865 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:58.782870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:58.783022 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 05:40:58.783171 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 05:40:58.783487 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 05:40:58.785235 32330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:40:58.786377 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.785983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60374)
0809 05:40:58.786158 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.786251 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 05:40:58.787391 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.787048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60382)
0809 05:40:58.787208 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.787292 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 05:40:58.787619 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:58.798244 32439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 2
I20260809 05:40:58.799401 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.798946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60394)
0809 05:40:58.799141 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.799243 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 05:40:58.800422 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.800084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60406)
0809 05:40:58.800245 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.800310 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 05:40:58.800638 32439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:58.838891 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 1
I20260809 05:40:58.840195 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.839710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60408)
0809 05:40:58.839916 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.840030 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 05:40:58.841223 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.840874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60414)
0809 05:40:58.841027 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.841109 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 05:40:58.841450 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:58.891887 32439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 3
I20260809 05:40:58.893105 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.892592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60428)
0809 05:40:58.892821 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.892939 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 05:40:58.894213 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.893810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60438)
0809 05:40:58.894046 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.894112 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260809 05:40:58.894423 32439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:58.909674 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 2
I20260809 05:40:58.910988 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.910419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60454)
0809 05:40:58.910699 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.910811 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 05:40:58.912091 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:58.911704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60470)
0809 05:40:58.911868 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:58.911968 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 05:40:58.912305 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:59.018559 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 3
I20260809 05:40:59.019999 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.019391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60484)
0809 05:40:59.019651 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.019776 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 05:40:59.021366 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.020846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60488)
0809 05:40:59.021110 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.021224 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260809 05:40:59.021672 32330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
I20260809 05:40:59.037678 32439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 4
I20260809 05:40:59.039052 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.038483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60502)
0809 05:40:59.038777 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.038872 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 05:40:59.040150 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.039795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60512)
0809 05:40:59.039957 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.040037 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 05:40:59.040418 32439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:36893: Network error: Client connection negotiation failed: client connection to 127.30.137.252:36893: connect: Connection refused (error 111)
W20260809 05:40:59.075501 32452 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:40:59.075865 32452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:40:59.075946 32452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:40:59.076031 32452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:40:59.083710 32452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:40:59.083817 32452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:40:59.083877 32452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:40:59.083931 32452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:40:59.083993 32452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:40:59.094219 32452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:35523,127.30.137.253:38235,127.30.137.252:36893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab
--principal=kudu/127.30.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:36893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:40:59.095722 32452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:40:59.097152 32452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:40:59.109439 32457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:59.109468 32458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:40:59.109386 32461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:40:59.110848 32452 server_base.cc:1034] running on GCE node
I20260809 05:40:59.111689 32452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:40:59.113175 32452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:40:59.114444 32452 hybrid_clock.cc:648] HybridClock initialized: now 1786254059114368 us; error 46 us; skew 500 ppm
I20260809 05:40:59.114864 32452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:40:59.118268 32452 init.cc:377] Logged in from keytab as kudu/127.30.137.252@KRBTEST.COM (short username kudu)
I20260809 05:40:59.120004 32452 webserver.cc:460] Webserver started at http://127.30.137.252:44857/ using document root <none> and password file <none>
I20260809 05:40:59.120775 32452 fs_manager.cc:359] Metadata directory not provided
I20260809 05:40:59.120929 32452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:40:59.121279 32452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:40:59.124593 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data/instance:
uuid: "a69f40f64d8741f3b3cc49e04e7f1756"
format_stamp: "Formatted at 2026-08-09 05:40:59 on dist-test-slave-3kk6"
server_key: "5fcce89f6495c52e0515b89af2edcbff"
server_key_iv: "6fe9fb46262fe688fe158c33c0f82ab0"
server_key_version: "encryptionkey@0"
I20260809 05:40:59.125495 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal/instance:
uuid: "a69f40f64d8741f3b3cc49e04e7f1756"
format_stamp: "Formatted at 2026-08-09 05:40:59 on dist-test-slave-3kk6"
server_key: "5fcce89f6495c52e0515b89af2edcbff"
server_key_iv: "6fe9fb46262fe688fe158c33c0f82ab0"
server_key_version: "encryptionkey@0"
I20260809 05:40:59.131387 32452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 05:40:59.135537 32472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:40:59.137069 32452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260809 05:40:59.137298 32452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal
uuid: "a69f40f64d8741f3b3cc49e04e7f1756"
format_stamp: "Formatted at 2026-08-09 05:40:59 on dist-test-slave-3kk6"
server_key: "5fcce89f6495c52e0515b89af2edcbff"
server_key_iv: "6fe9fb46262fe688fe158c33c0f82ab0"
server_key_version: "encryptionkey@0"
I20260809 05:40:59.137557 32452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:40:59.172089 32452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:40:59.177047 32452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:40:59.177371 32452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:40:59.197640 32452 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.252:36893
I20260809 05:40:59.197661 32550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.252:36893 every 8 connection(s)
I20260809 05:40:59.198913 32330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 4
I20260809 05:40:59.199285 32452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
I20260809 05:40:59.200670 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 32452
I20260809 05:40:59.200893 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786253675866932-31271-0/minicluster-data/master-2/wal/instance
I20260809 05:40:59.201488 31271 external_mini_cluster.cc:1428] Setting key 75e6c2b54ebfef042f3f92b0d8c7e1d5
I20260809 05:40:59.204993 32553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:40:59.219524 32553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:59.238835 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.199661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60518)
0809 05:40:59.199908 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.199937 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:59.200098 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:59.216692 (+ 16594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:59.216699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:59.217288 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:59.217514 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.217527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.219991 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.219997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.221355 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.221372 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.222862 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.222869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.223114 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.224045 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:59.224078 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:59.228773 (+ 4695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:59.235415 (+ 6642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.235424 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.235431 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.236014 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.236538 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.236543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.236546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.236720 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.237662 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:59.237667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:59.237839 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 05:40:59.237995 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 05:40:59.238339 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 05:40:59.240490 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.201056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.30.137.252:36893)
0809 05:40:59.211332 (+ 10276us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.211350 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.211424 (+ 74us) server_negotiation.cc:407] Connection header received
0809 05:40:59.215587 (+ 4163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.215624 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.216205 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.217696 (+ 1491us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.217907 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.219779 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.221480 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.222693 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.222935 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.229116 (+ 6181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.229128 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.229143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.229198 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.235227 (+ 6029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.236186 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.236193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.236196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.236386 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.236863 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.236868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.236875 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.237495 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.237681 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.238588 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.239198 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":363,"server-negotiator.queue_time_us":9993,"spinlock_wait_cycles":4864,"thread_start_us":9709,"threads_started":1}
I20260809 05:40:59.240588 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.211303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.30.137.252:36893)
0809 05:40:59.212750 (+ 1447us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.212758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.212790 (+ 32us) server_negotiation.cc:407] Connection header received
0809 05:40:59.215624 (+ 2834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.215637 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.216201 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.216666 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.218353 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.219697 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.221390 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.223333 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.223477 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.229116 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.229128 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.229143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.229204 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.235227 (+ 6023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.236094 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.236103 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.236113 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.236375 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.236813 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.236818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.236822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.237503 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.238946 (+ 1443us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.239068 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.239329 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":926,"server-negotiator.queue_time_us":1292,"spinlock_wait_cycles":54912,"thread_start_us":1092,"threads_started":1}
I20260809 05:40:59.245270 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.222741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:35523 (local address 127.0.0.1:42862)
0809 05:40:59.227512 (+ 4771us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.227563 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:59.227790 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:59.228217 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:59.228227 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:59.229134 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:59.229474 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.229483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.230908 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.230916 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.232414 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.232427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.233350 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.233358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.233494 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.234139 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:59.234158 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:59.237221 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:59.241895 (+ 4674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.241906 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.241916 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.242587 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.243081 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.243087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.243090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.243277 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.244017 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:59.244025 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:59.244226 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 05:40:59.244446 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 05:40:59.244803 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4512,"thread_start_us":4394,"threads_started":1}
I20260809 05:40:59.246340 32562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.222618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.30.137.254:35523)
0809 05:40:59.226996 (+ 4378us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.227001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.227680 (+ 679us) server_negotiation.cc:407] Connection header received
0809 05:40:59.227921 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.227929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.228019 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.228133 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.229634 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.230641 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.232581 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.233032 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.233180 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.237448 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.237457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.237462 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.237499 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.241722 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.242778 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.242784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.242788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.242946 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.243412 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.243417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.243420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.243842 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.244060 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.245604 (+ 1544us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.245878 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4235,"thread_start_us":2033,"threads_started":1}
I20260809 05:40:59.248729 32330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: Bootstrap starting.
I20260809 05:40:59.252547 32330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:59.253144 32553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:40:59.253974 32330 log.cc:824] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: Log is configured to *not* fsync() on all Append() calls
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:40:59.262315 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.244567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42864 (local address 127.30.137.254:35523)
0809 05:40:59.245788 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.245792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.245814 (+ 22us) server_negotiation.cc:407] Connection header received
0809 05:40:59.245861 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.245864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.245922 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.246072 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.247600 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.248502 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.249905 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.250426 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.250590 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.255991 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.255996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.256000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.256033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.259401 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.260133 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.260140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.260145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.260312 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.260708 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.260714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.260717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.261038 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.261356 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.261683 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.261928 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":497,"threads_started":1}
I20260809 05:40:59.263990 32330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: No bootstrap required, opened a new log
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:40:59.275102 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.254177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:38235 (local address 127.0.0.1:47890)
0809 05:40:59.254417 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.254442 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:59.254573 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:59.255125 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:59.255134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:59.255810 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:59.256089 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.256104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.257495 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.257504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.259408 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.259431 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.261024 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.261031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.261245 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.262716 (+ 1471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:59.262741 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:59.266309 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:59.271344 (+ 5035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.271356 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.271363 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.272223 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.272792 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.272801 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.272808 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.273054 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.273763 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:59.273771 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:59.274015 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 05:40:59.274226 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 05:40:59.274698 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 05:40:59.275168 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.254522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.30.137.253:38235)
0809 05:40:59.254741 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.254748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.254767 (+ 19us) server_negotiation.cc:407] Connection header received
0809 05:40:59.254842 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.254849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.254945 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.255056 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.256307 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.257298 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.259647 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.260591 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.260709 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.266590 (+ 5881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.266598 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.266603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.266712 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.271110 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.272418 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.272426 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.272430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.272634 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.273199 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.273205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.273208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.273621 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.273786 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.274353 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.274600 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:40:59.275648 32330 raft_consensus.cc:348] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.276151 32330 raft_consensus.cc:729] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c454cd78df4141cc9fde76a38fb489fb, State: Initialized, Role: FOLLOWER
I20260809 05:40:59.276232 32553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:40:59.276751 32330 consensus_queue.cc:260] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.279584 32567 sys_catalog.cc:455] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.280114 32567 sys_catalog.cc:458] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:40:59.281019 32330 sys_catalog.cc:564] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:40:59.294417 32563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.277200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60532)
0809 05:40:59.277346 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.277362 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:59.277443 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:59.277860 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:59.277863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:59.278291 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:59.278458 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.278467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.280024 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.280030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.281349 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.281363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.282045 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.282050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.282289 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.283129 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:59.283160 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:59.286804 (+ 3644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:59.291214 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.291220 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.291224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.291847 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.292297 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.292303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.292306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.292458 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.293292 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:59.293299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:59.293456 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 05:40:59.293640 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 05:40:59.293988 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":23}
W20260809 05:40:59.294971 32269 tablet.cc:2369] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:59.295358 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.277097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.30.137.252:36893)
0809 05:40:59.277546 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.277554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.277571 (+ 17us) server_negotiation.cc:407] Connection header received
0809 05:40:59.277623 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.277631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.277732 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.277838 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.278605 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.279888 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.281485 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.281897 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.282034 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.287004 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.287011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.287016 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.287058 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.291046 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.291991 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.291999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.292005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.292165 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.292576 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.292583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.292588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.293072 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.293236 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.294767 (+ 1531us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.295011 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":302}
I20260809 05:40:59.299641 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.264376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47904 (local address 127.30.137.253:38235)
0809 05:40:59.275230 (+ 10854us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.275236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.277125 (+ 1889us) server_negotiation.cc:407] Connection header received
0809 05:40:59.277186 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.277194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.277278 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.277401 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.279635 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.280920 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.283410 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.284140 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.284295 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.290527 (+ 6232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.290538 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.290545 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.290594 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.294576 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.295974 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.295979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.295982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.296313 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.296834 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.296839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.296842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.297432 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.298795 (+ 1363us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.298929 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.299216 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10685,"thread_start_us":10439,"threads_started":1}
W20260809 05:40:59.301406 32582 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 05:40:59.301519 32553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: Bootstrap starting.
W20260809 05:40:59.301595 32582 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:40:59.305337 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:40:59.305914 32553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:59.307330 32553 log.cc:824] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: Log is configured to *not* fsync() on all Append() calls
I20260809 05:40:59.292964 32235 master_runner.cc:186] Error getting master registration for 127.30.137.253:38235: OK, instance_id { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" instance_seqno: 1786254058610939 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:40:59.309087 32235 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:40:59.309417 32296 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c454cd78df4141cc9fde76a38fb489fb"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42864
I20260809 05:40:59.310263 32296 raft_consensus.cc:482] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 05:40:59.310547 32296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:59.313035 32553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: No bootstrap required, opened a new log
I20260809 05:40:59.314294 32296 raft_consensus.cc:504] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.316959 32296 leader_election.cc:288] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [CANDIDATE]: Term 1 election: Requested vote from peers d8b504da920c442495dedf1bc687c22c (127.30.137.253:38235), a69f40f64d8741f3b3cc49e04e7f1756 (127.30.137.252:36893)
I20260809 05:40:59.324927 32553 raft_consensus.cc:348] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.325435 32553 raft_consensus.cc:729] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a69f40f64d8741f3b3cc49e04e7f1756, State: Initialized, Role: FOLLOWER
I20260809 05:40:59.326013 32553 consensus_queue.cc:260] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.339650 32439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } attempt: 5
I20260809 05:40:59.340947 32553 sys_catalog.cc:564] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:59.340971 32509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c454cd78df4141cc9fde76a38fb489fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a69f40f64d8741f3b3cc49e04e7f1756"
I20260809 05:40:59.341969 32509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:59.348934 32509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c454cd78df4141cc9fde76a38fb489fb in term 1.
I20260809 05:40:59.339483 32590 sys_catalog.cc:455] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.350096 32590 sys_catalog.cc:458] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 05:40:59 dist-test-slave-3kk6 krb5kdc[32212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
I20260809 05:40:59.364094 32264 leader_election.cc:302] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a69f40f64d8741f3b3cc49e04e7f1756, c454cd78df4141cc9fde76a38fb489fb; no voters:
I20260809 05:40:59.365021 32567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:40:59.376787 32567 raft_consensus.cc:686] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [term 1 LEADER]: Becoming Leader. State: Replica: c454cd78df4141cc9fde76a38fb489fb, State: Running, Role: LEADER
I20260809 05:40:59.377847 32567 consensus_queue.cc:237] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.387285 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.341560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.30.137.252:36893)
0809 05:40:59.341788 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.341797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.341822 (+ 25us) server_negotiation.cc:407] Connection header received
0809 05:40:59.341950 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.341958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.342091 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.342229 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.343732 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.344888 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.347129 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.348119 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.348278 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.356206 (+ 7928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.356220 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.356228 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.356294 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.361740 (+ 5446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.375183 (+ 13443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.375194 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.375201 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.375512 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.382637 (+ 7125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.382693 (+ 56us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.382699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.383313 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.383510 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.386200 (+ 2690us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.386589 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":80}
I20260809 05:40:59.392463 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.340580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:36893 (local address 127.0.0.1:60534)
0809 05:40:59.341072 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 05:40:59.341100 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 05:40:59.341258 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:40:59.342534 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:40:59.342541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:40:59.343207 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:40:59.343479 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.343498 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.345112 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.345121 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.346881 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:40:59.346905 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.348628 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.348637 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:40:59.349092 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.350337 (+ 1245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:40:59.350374 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:40:59.355713 (+ 5339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:40:59.362232 (+ 6519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.362242 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.362250 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.374618 (+ 12368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.376090 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:40:59.376102 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:40:59.376163 (+ 61us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:40:59.382160 (+ 5997us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:40:59.383981 (+ 1821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:40:59.383989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:40:59.385070 (+ 1081us) client_negotiation.cc:769] Sending connection context
0809 05:40:59.388256 (+ 3186us) client_negotiation.cc:240] Negotiation successful
0809 05:40:59.390058 (+ 1802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319}
I20260809 05:40:59.401721 32439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: Bootstrap starting.
I20260809 05:40:59.405920 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.351821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.30.137.253:38235)
0809 05:40:59.355093 (+ 3272us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.355104 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.375373 (+ 20269us) server_negotiation.cc:407] Connection header received
0809 05:40:59.377229 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.377237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.377327 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.377411 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.384223 (+ 6812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.385505 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.391951 (+ 6446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.392634 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.392749 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.398049 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.398057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.398064 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.398110 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.402235 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.403260 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.403268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.403273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.403438 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.403945 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.403951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.403954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.404350 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.404631 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.405258 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.405510 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2592}
I20260809 05:40:59.408409 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.359843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.30.137.252:36893)
0809 05:40:59.360075 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.360084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.382115 (+ 22031us) server_negotiation.cc:407] Connection header received
0809 05:40:59.382637 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.382644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.382755 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.382841 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.388722 (+ 5881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.389926 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.396543 (+ 6617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.396999 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.397128 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.400939 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.400948 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.400953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.401004 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.404927 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.405772 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.405778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.405781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.405949 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.406422 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.406428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.406431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.406871 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.407164 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.407777 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.408046 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 05:40:59.410979 32439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: Neither blocks nor log segments found. Creating new log.
I20260809 05:40:59.411919 32564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:40:59.345720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.30.137.254:35523)
0809 05:40:59.345998 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.346007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.367305 (+ 21298us) server_negotiation.cc:407] Connection header received
0809 05:40:59.367727 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.367738 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.367854 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.367951 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.371167 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.372692 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.375264 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:40:59.376131 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:40:59.376256 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:40:59.382277 (+ 6021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:40:59.382294 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:40:59.382301 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:40:59.382353 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:40:59.386552 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.396411 (+ 9859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.396417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.396421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.409351 (+ 12930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:40:59.409944 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:40:59.409952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:40:59.409955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:40:59.410371 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:40:59.410642 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 05:40:59.411216 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 05:40:59.411479 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:40:59.417685 32602 sys_catalog.cc:455] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c454cd78df4141cc9fde76a38fb489fb. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.418177 32602 sys_catalog.cc:458] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:59.419211 32439 log.cc:824] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: Log is configured to *not* fsync() on all Append() calls
W20260809 05:40:59.429807 32481 tablet.cc:2369] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:40:59.435324 32439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: No bootstrap required, opened a new log
I20260809 05:40:59.437552 32610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 05:40:59.447016 32610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:40:59.451715 32439 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.452549 32439 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b504da920c442495dedf1bc687c22c, State: Initialized, Role: FOLLOWER
I20260809 05:40:59.453344 32439 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } }
I20260809 05:40:59.460901 32621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.464159 32621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:40:59.463943 32439 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:40:59.465858 32405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c454cd78df4141cc9fde76a38fb489fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d8b504da920c442495dedf1bc687c22c"
I20260809 05:40:59.466471 32405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:40:59.469550 32626 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 05:40:59.470422 32405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c454cd78df4141cc9fde76a38fb489fb in term 1.
I20260809 05:40:59.484578 32509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [term 1 FOLLOWER]: Refusing update from remote peer c454cd78df4141cc9fde76a38fb489fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:59.487634 32405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [term 1 FOLLOWER]: Refusing update from remote peer c454cd78df4141cc9fde76a38fb489fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:40:59.488379 32602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:59.490005 32602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:40:59.478525 32452 master_runner.cc:186] Error getting master registration for 127.30.137.253:38235: OK, instance_id { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" instance_seqno: 1786254058610939 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:40:59.501101 32452 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:40:59.517649 32590 sys_catalog.cc:455] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c454cd78df4141cc9fde76a38fb489fb. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.518102 32590 sys_catalog.cc:458] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:40:59.526924 32624 mvcc.cc:205] Tried to move back new op lower bound from 7316496627622449152 to 7316496627238043648. Current Snapshot: MvccSnapshot[applied={T|T < 7316496627622449152}]
I20260809 05:40:59.536594 32621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c454cd78df4141cc9fde76a38fb489fb. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.537108 32621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:40:59.541052 32567 sys_catalog.cc:455] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.541474 32567 sys_catalog.cc:458] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:59.541815 32567 sys_catalog.cc:455] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.542095 32567 sys_catalog.cc:458] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb [sys.catalog]: This master's current role is: LEADER
I20260809 05:40:59.543543 32590 sys_catalog.cc:455] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.543897 32590 sys_catalog.cc:458] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:40:59.549365 32610 catalog_manager.cc:1347] Generated new cluster ID: 6657a3e83dfc4bbbb6666a9274512c95
I20260809 05:40:59.549533 32610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:40:59.551280 32621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c454cd78df4141cc9fde76a38fb489fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c454cd78df4141cc9fde76a38fb489fb" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 35523 } } peers { permanent_uuid: "d8b504da920c442495dedf1bc687c22c" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 38235 } } peers { permanent_uuid: "a69f40f64d8741f3b3cc49e04e7f1756" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 36893 } } }
I20260809 05:40:59.551620 32621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:40:59.585191 32656 catalog_manager.cc:1259] Loaded cluster ID: 6657a3e83dfc4bbbb6666a9274512c95
I20260809 05:40:59.585331 32656 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: loading cluster ID for follower catalog manager: success
W20260809 05:40:59.586684 32656 catalog_manager.cc:1576] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 05:40:59.586854 32656 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:40:59.610077 32610 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:40:59.615253 32610 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:40:59.655645 32610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c454cd78df4141cc9fde76a38fb489fb: Generated new TSK 0
I20260809 05:40:59.657073 32610 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 05:40:59.774434 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.756139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.30.137.253:38235)
0809 05:40:59.756372 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.756378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.771812 (+ 15434us) server_negotiation.cc:407] Connection header received
0809 05:40:59.772234 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.772245 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.772361 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.772463 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.774134 (+ 1671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47910: BlockingRecv error: recv got EOF from 127.0.0.1:47910 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260809 05:40:59.776079 32564 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.756141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.30.137.254:35523)
0809 05:40:59.756376 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.756382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.773778 (+ 17396us) server_negotiation.cc:407] Connection header received
0809 05:40:59.774279 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.774285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.774393 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.774485 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.775822 (+ 1337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42886: BlockingRecv error: recv got EOF from 127.0.0.1:42886 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 05:40:59.776858 32558 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.762922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.30.137.252:36893)
0809 05:40:59.763183 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.763192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.774193 (+ 11001us) server_negotiation.cc:407] Connection header received
0809 05:40:59.774493 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.774500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.774595 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.774859 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.776610 (+ 1751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60546: BlockingRecv error: recv got EOF from 127.0.0.1:60546 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 05:40:59.792083 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.789431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.30.137.253:38235)
0809 05:40:59.789626 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.789633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.790210 (+ 577us) server_negotiation.cc:407] Connection header received
0809 05:40:59.790473 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.790480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.790702 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.790831 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.791806 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47916: BlockingRecv error: recv got EOF from 127.0.0.1:47916 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 05:40:59.802073 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.799727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.30.137.253:38235)
0809 05:40:59.799948 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.799959 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.800255 (+ 296us) server_negotiation.cc:407] Connection header received
0809 05:40:59.800479 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.800485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.800583 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.800717 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.801764 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47918: BlockingRecv error: recv got EOF from 127.0.0.1:47918 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 05:40:59.812109 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.809589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.30.137.253:38235)
0809 05:40:59.809790 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.809797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.810372 (+ 575us) server_negotiation.cc:407] Connection header received
0809 05:40:59.810599 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.810605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.810731 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.810841 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.811828 (+ 987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47922: BlockingRecv error: recv got EOF from 127.0.0.1:47922 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 05:40:59.822793 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.820103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.30.137.253:38235)
0809 05:40:59.820361 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.820368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.821038 (+ 670us) server_negotiation.cc:407] Connection header received
0809 05:40:59.821245 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.821252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.821359 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.821502 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.822441 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47928: BlockingRecv error: recv got EOF from 127.0.0.1:47928 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
W20260809 05:40:59.837587 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.835016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47934 (local address 127.30.137.253:38235)
0809 05:40:59.835245 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.835251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.835877 (+ 626us) server_negotiation.cc:407] Connection header received
0809 05:40:59.836113 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.836121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.836239 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.836376 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.837334 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47934: BlockingRecv error: recv got EOF from 127.0.0.1:47934 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 05:40:59.860762 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.858100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.30.137.253:38235)
0809 05:40:59.858334 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.858342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.859011 (+ 669us) server_negotiation.cc:407] Connection header received
0809 05:40:59.859249 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.859255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.859362 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.859479 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.860440 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47938: BlockingRecv error: recv got EOF from 127.0.0.1:47938 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260809 05:40:59.899946 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.897227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.30.137.253:38235)
0809 05:40:59.897461 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.897468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.898099 (+ 631us) server_negotiation.cc:407] Connection header received
0809 05:40:59.898389 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.898398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.898529 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.898682 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.899643 (+ 961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47946: BlockingRecv error: recv got EOF from 127.0.0.1:47946 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 05:40:59.971537 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:40:59.969001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.30.137.253:38235)
0809 05:40:59.969239 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 05:40:59.969246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:40:59.969674 (+ 428us) server_negotiation.cc:407] Connection header received
0809 05:40:59.969867 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:40:59.969874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:40:59.969996 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:40:59.970140 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:40:59.971265 (+ 1125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47956: BlockingRecv error: recv got EOF from 127.0.0.1:47956 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260809 05:41:00.106587 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:00.104383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.30.137.253:38235)
0809 05:41:00.104596 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.104601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.105023 (+ 422us) server_negotiation.cc:407] Connection header received
0809 05:41:00.105274 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.105282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.105408 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.105552 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:00.106387 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47962: BlockingRecv error: recv got EOF from 127.0.0.1:47962 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 05:41:00.368521 32566 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:00.366577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.30.137.253:38235)
0809 05:41:00.366825 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.366830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.367130 (+ 300us) server_negotiation.cc:407] Connection header received
0809 05:41:00.367320 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.367325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.367417 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.367520 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:00.368299 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47964: BlockingRecv error: recv got EOF from 127.0.0.1:47964 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
I20260809 05:41:00.474359 32626 catalog_manager.cc:1259] Loaded cluster ID: 6657a3e83dfc4bbbb6666a9274512c95
I20260809 05:41:00.474521 32626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: loading cluster ID for follower catalog manager: success
I20260809 05:41:00.477272 32626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: acquiring CA information for follower catalog manager: success
I20260809 05:41:00.479637 32626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a69f40f64d8741f3b3cc49e04e7f1756: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:41:00.591624 32656 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: acquiring CA information for follower catalog manager: success
I20260809 05:41:00.594331 32656 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8b504da920c442495dedf1bc687c22c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:41:00.890303 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:00.884233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.30.137.253:38235)
0809 05:41:00.884821 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.884825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.885014 (+ 189us) server_negotiation.cc:407] Connection header received
0809 05:41:00.885176 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.885181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.885341 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.885457 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:00.885917 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.886745 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.888098 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.888519 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.888617 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:00.889106 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:00.889631 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:00.889742 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:00.889878 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 05:41:00.889958 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":339,"threads_started":1}
I20260809 05:41:00.903162 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:00.897038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.30.137.252:36893)
0809 05:41:00.897614 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.897619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.897639 (+ 20us) server_negotiation.cc:407] Connection header received
0809 05:41:00.897834 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.897839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.898012 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.898174 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:00.898557 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.899427 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.900745 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.901302 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.901426 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:00.901934 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:00.902501 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:00.902617 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:00.902783 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 05:41:00.902862 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":321,"threads_started":1}
I20260809 05:41:00.927497 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:00.912217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.30.137.254:35523)
0809 05:41:00.920377 (+ 8160us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.920387 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.920418 (+ 31us) server_negotiation.cc:407] Connection header received
0809 05:41:00.920583 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.920591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.920853 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.921099 (+ 246us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:00.921752 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.922860 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.925056 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.925603 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.925766 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:00.926271 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:00.926819 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:00.926942 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:00.927086 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 05:41:00.927180 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7982,"thread_start_us":7784,"threads_started":1}
I20260809 05:41:00.927634 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:00.912703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47986 (local address 127.30.137.253:38235)
0809 05:41:00.912956 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.912965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.920902 (+ 7937us) server_negotiation.cc:407] Connection header received
0809 05:41:00.921246 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.921253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.921356 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.921425 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:00.922268 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.923409 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.925348 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.926026 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.926152 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:00.926790 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:00.926922 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:00.926990 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:00.927228 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 05:41:00.927330 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 05:41:00.928045 304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:00.912687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60560 (local address 127.30.137.252:36893)
0809 05:41:00.912936 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:00.912945 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:00.921100 (+ 8155us) server_negotiation.cc:407] Connection header received
0809 05:41:00.921474 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:00.921480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:00.921578 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:00.921680 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:00.922710 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.923750 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.925800 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:00.926463 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:00.926566 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:00.927250 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:00.927374 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:00.927439 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:00.927671 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 05:41:00.927771 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 05:41:00.933682 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32235
I20260809 05:41:00.934880 32325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:41:01.050460 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32235
I20260809 05:41:01.084129 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32339
I20260809 05:41:01.085454 32432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:41:01.198304 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32339
I20260809 05:41:01.226336 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 32452
I20260809 05:41:01.227809 32540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:41:01.341204 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 32452
2026-08-09T05:41:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5499 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 05:41:01 dist-test-slave-3kk6 krb5kdc[324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 05:41:01 dist-test-slave-3kk6 krb5kdc[324](info): set up 2 sockets
Aug 09 05:41:01 dist-test-slave-3kk6 krb5kdc[324](info): commencing operation
krb5kdc: starting...
W20260809 05:41:02.733886 31271 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.003s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/test-user.keytab.
Aug 09 05:41:02 dist-test-slave-3kk6 krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T05:41:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T05:41:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.137.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
Entry for principal kudu/127.30.137.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab.
I20260809 05:41:03.120217 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:37629
--webserver_interface=127.30.137.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:37629,127.30.137.253:41433,127.30.137.252:42307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 05:41:03.497526 348 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:41:03.497977 348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:41:03.498142 348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:41:03.498275 348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:41:03.506134 348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:41:03.506266 348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:41:03.506364 348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:41:03.506451 348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:41:03.506573 348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:41:03.517256 348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:37629,127.30.137.253:41433,127.30.137.252:42307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.254.keytab
--principal=kudu/127.30.137.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.254:37629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.137.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:41:03.518780 348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:41:03.520267 348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:41:03.533095 357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:41:03.532997 359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:41:03.533053 362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:41:03.534330 348 server_base.cc:1034] running on GCE node
I20260809 05:41:03.535336 348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:41:03.536873 348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:41:03.538125 348 hybrid_clock.cc:648] HybridClock initialized: now 1786254063538054 us; error 37 us; skew 500 ppm
I20260809 05:41:03.538511 348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:41:03 dist-test-slave-3kk6 krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254063, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:41:03.547509 348 init.cc:377] Logged in from keytab as kudu/127.30.137.254@KRBTEST.COM (short username kudu)
I20260809 05:41:03.549464 348 webserver.cc:460] Webserver started at http://127.30.137.254:42957/ using document root <none> and password file <none>
I20260809 05:41:03.550274 348 fs_manager.cc:359] Metadata directory not provided
I20260809 05:41:03.550379 348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:41:03.550776 348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:41:03.554160 348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data/instance:
uuid: "ca9e2a1da2824caab42b39dd4d3f5799"
format_stamp: "Formatted at 2026-08-09 05:41:03 on dist-test-slave-3kk6"
server_key: "2304f5d6113f785d033fe32cbbd24764"
server_key_iv: "f3d2baf11c93a03c2a3448eb6b1a73e2"
server_key_version: "encryptionkey@0"
I20260809 05:41:03.555162 348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal/instance:
uuid: "ca9e2a1da2824caab42b39dd4d3f5799"
format_stamp: "Formatted at 2026-08-09 05:41:03 on dist-test-slave-3kk6"
server_key: "2304f5d6113f785d033fe32cbbd24764"
server_key_iv: "f3d2baf11c93a03c2a3448eb6b1a73e2"
server_key_version: "encryptionkey@0"
I20260809 05:41:03.561148 348 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 05:41:03.565451 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:41:03.567181 348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 05:41:03.567413 348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal
uuid: "ca9e2a1da2824caab42b39dd4d3f5799"
format_stamp: "Formatted at 2026-08-09 05:41:03 on dist-test-slave-3kk6"
server_key: "2304f5d6113f785d033fe32cbbd24764"
server_key_iv: "f3d2baf11c93a03c2a3448eb6b1a73e2"
server_key_version: "encryptionkey@0"
I20260809 05:41:03.567687 348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:41:03.586908 348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:41:03.591456 348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:41:03.591786 348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:41:03.612179 348 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.254:37629
I20260809 05:41:03.612187 453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.254:37629 every 8 connection(s)
I20260809 05:41:03.613694 348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/data/info.pb
I20260809 05:41:03.617724 455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:41:03.620297 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 348
I20260809 05:41:03.620507 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-0/wal/instance
I20260809 05:41:03.620992 31271 external_mini_cluster.cc:1428] Setting key 092edffc3b1552772915c90691f86d4e
I20260809 05:41:03.624727 455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:41:03 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254063, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:41:03.649044 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.628256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:37629 (local address 127.0.0.1:37300)
0809 05:41:03.629445 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.629488 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:03.631414 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:03.632313 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:03.632320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:03.633071 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:03.633429 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:03.633446 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:03.635122 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:03.635133 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:03.636215 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:03.636227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:03.637209 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:03.637216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:03.637471 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:03.638198 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:03.638223 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:03.640987 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:03.644552 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:03.644566 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:03.644580 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:03.644996 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:03.645404 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:03.645411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:03.645417 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:03.645584 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:03.646394 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:03.646401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:03.646615 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 05:41:03.646936 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 05:41:03.647562 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":142,"thread_start_us":758,"threads_started":1}
I20260809 05:41:03.649062 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.628223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.30.137.254:37629)
0809 05:41:03.629137 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:03.629156 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:03.629619 (+ 463us) server_negotiation.cc:407] Connection header received
0809 05:41:03.631814 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:03.631820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:03.632060 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:03.632274 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:03.633655 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:03.634960 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:03.636369 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:03.636948 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:03.637154 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:03.641231 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:03.641240 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:03.641254 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:03.641286 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:03.644409 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:03.645132 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:03.645145 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:03.645147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:03.645295 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:03.645695 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:03.645702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:03.645705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:03.646241 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:03.646462 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:03.647056 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 05:41:03.647437 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":647,"thread_start_us":405,"threads_started":1}
I20260809 05:41:03.652872 455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:41:03.654161 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.653731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39268)
0809 05:41:03.653920 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.654036 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 05:41:03.654445 378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
I20260809 05:41:03.656311 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.655940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39280)
0809 05:41:03.656103 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.656178 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 05:41:03.656502 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:41433: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.137.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab.
Entry for principal kudu/127.30.137.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab.
I20260809 05:41:03.705972 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:41433
--webserver_interface=127.30.137.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab
--principal=kudu/127.30.137.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:37629,127.30.137.253:41433,127.30.137.252:42307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:41:03.705873 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } attempt: 1
I20260809 05:41:03.707283 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.706795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39286)
0809 05:41:03.707009 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.707143 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:41:03.708391 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.707977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39290)
0809 05:41:03.708132 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.708208 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 05:41:03.708593 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:41433: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
I20260809 05:41:03.776842 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } attempt: 2
I20260809 05:41:03.778278 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.777744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39294)
0809 05:41:03.778013 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.778115 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 05:41:03.779521 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.779133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39304)
0809 05:41:03.779288 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.779377 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 05:41:03.779812 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:41433: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
I20260809 05:41:03.871095 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } attempt: 3
I20260809 05:41:03.872560 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.872043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39320)
0809 05:41:03.872307 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.872415 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 05:41:03.873723 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:03.873315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39330)
0809 05:41:03.873476 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:03.873557 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 05:41:03.873922 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:41433: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
I20260809 05:41:04.017221 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } attempt: 4
I20260809 05:41:04.018611 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.018018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39342)
0809 05:41:04.018371 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.018468 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260809 05:41:04.019814 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.019429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39348)
0809 05:41:04.019594 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.019692 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 05:41:04.020004 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.253:41433: Network error: Client connection negotiation failed: client connection to 127.30.137.253:41433: connect: Connection refused (error 111)
W20260809 05:41:04.066406 468 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:41:04.066898 468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:41:04.067029 468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:41:04.067188 468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:41:04.075109 468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:41:04.075268 468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:41:04.075404 468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:41:04.075496 468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:41:04.075611 468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:41:04.086231 468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:37629,127.30.137.253:41433,127.30.137.252:42307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.253.keytab
--principal=kudu/127.30.137.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.253:41433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.137.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:41:04.087770 468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:41:04.089211 468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:41:04.102792 476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:41:04.103004 477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:41:04.104470 479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:41:04.104804 468 server_base.cc:1034] running on GCE node
I20260809 05:41:04.105711 468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:41:04.107322 468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:41:04.108552 468 hybrid_clock.cc:648] HybridClock initialized: now 1786254064108498 us; error 35 us; skew 500 ppm
I20260809 05:41:04.108930 468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:41:04.112598 468 init.cc:377] Logged in from keytab as kudu/127.30.137.253@KRBTEST.COM (short username kudu)
I20260809 05:41:04.114215 468 webserver.cc:460] Webserver started at http://127.30.137.253:44413/ using document root <none> and password file <none>
I20260809 05:41:04.115062 468 fs_manager.cc:359] Metadata directory not provided
I20260809 05:41:04.115160 468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:41:04.115453 468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:41:04.118495 468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data/instance:
uuid: "2e83fcbd087641c6abae8544b66a2ddd"
format_stamp: "Formatted at 2026-08-09 05:41:04 on dist-test-slave-3kk6"
server_key: "5f9bdef7f1aa9d71173598621e2057ec"
server_key_iv: "f95d745ec9acef525d82ff867ff4f268"
server_key_version: "encryptionkey@0"
I20260809 05:41:04.119424 468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal/instance:
uuid: "2e83fcbd087641c6abae8544b66a2ddd"
format_stamp: "Formatted at 2026-08-09 05:41:04 on dist-test-slave-3kk6"
server_key: "5f9bdef7f1aa9d71173598621e2057ec"
server_key_iv: "f95d745ec9acef525d82ff867ff4f268"
server_key_version: "encryptionkey@0"
I20260809 05:41:04.124914 468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 05:41:04.128676 489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:41:04.130105 468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 05:41:04.130321 468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal
uuid: "2e83fcbd087641c6abae8544b66a2ddd"
format_stamp: "Formatted at 2026-08-09 05:41:04 on dist-test-slave-3kk6"
server_key: "5f9bdef7f1aa9d71173598621e2057ec"
server_key_iv: "f95d745ec9acef525d82ff867ff4f268"
server_key_version: "encryptionkey@0"
I20260809 05:41:04.130532 468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:41:04.148726 468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:41:04.154263 468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:41:04.154584 468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:41:04.175401 468 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.253:41433
I20260809 05:41:04.175419 568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.253:41433 every 8 connection(s)
I20260809 05:41:04.177028 468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/data/info.pb
I20260809 05:41:04.181285 571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:41:04.183861 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 468
I20260809 05:41:04.184093 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-1/wal/instance
I20260809 05:41:04.184603 31271 external_mini_cluster.cc:1428] Setting key 75b1f4dddb80b75b3d1fb248340a7dc6
I20260809 05:41:04.189154 571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:41:04.215911 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.192279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.30.137.254:37629)
0809 05:41:04.192928 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.192944 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.194004 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 05:41:04.196892 (+ 2888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.196902 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.197002 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.197136 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.199233 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.200432 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.202276 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.202766 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.202911 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.207438 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.207447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.207454 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.207521 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.211818 (+ 4297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.212785 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.212795 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.212801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.212974 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.213498 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.213506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.213512 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.213964 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.214154 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.215090 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.215362 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":325,"threads_started":1}
I20260809 05:41:04.216702 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.192919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:37629 (local address 127.0.0.1:37312)
0809 05:41:04.193818 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.193876 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.196536 (+ 2660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.197382 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.197395 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.198183 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.198968 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.198994 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.200615 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.200622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.202067 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.202082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.203027 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.203035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.203292 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.204112 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.204139 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.207102 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.212035 (+ 4933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.212044 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.212060 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.212593 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.213130 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.213135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.213137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.213353 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.214134 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.214141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.214533 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.214932 (+ 399us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.215377 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":375,"threads_started":1}
I20260809 05:41:04.218750 571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.137.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.137.252@KRBTEST.COM" created.
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:41:04.235908 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.219644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39360)
0809 05:41:04.219849 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.219875 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.220004 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.220809 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.220813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.221290 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.221433 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.221443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.223004 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.223009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.223933 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.223945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.224855 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.224860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.225160 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.225902 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.225920 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.228109 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.232768 (+ 4659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.232772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.232775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.233152 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.233606 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.233610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.233612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.233781 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.234622 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.234626 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.234852 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.235043 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.235378 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":67}
I20260809 05:41:04.235961 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.219747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39360 (local address 127.30.137.253:41433)
0809 05:41:04.220396 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.220402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.220426 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:41:04.220470 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.220476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.220655 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.220816 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.221617 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.222865 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.224075 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.224633 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.224843 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.228360 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.228379 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.228402 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.228452 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.232591 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.233290 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.233302 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.233307 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.233462 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.233916 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.233923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.233929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.234468 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.234694 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.235257 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.235524 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":485,"thread_start_us":325,"threads_started":1}
I20260809 05:41:04.238941 571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:41:04.240294 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.239686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33840)
0809 05:41:04.239920 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.240062 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 05:41:04.240605 493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.242347 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.241955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33844)
0809 05:41:04.242156 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.242241 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 05:41:04.242815 571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.137.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab.
Entry for principal kudu/127.30.137.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab.
I20260809 05:41:04.269820 31271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
/tmp/dist-test-taskUEwy2v/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:42307
--webserver_interface=127.30.137.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab
--principal=kudu/127.30.137.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.137.212:46195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.137.254:37629,127.30.137.253:41433,127.30.137.252:42307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 05:41:04.292227 571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 1
I20260809 05:41:04.293591 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.293089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33848)
0809 05:41:04.293298 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.293418 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:41:04.294751 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.294336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33850)
0809 05:41:04.294498 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.294577 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 05:41:04.294970 571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.319276 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } attempt: 5
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254063, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:41:04.335299 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.320155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39370)
0809 05:41:04.320416 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.320441 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.320586 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.320991 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.320994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.321399 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.321596 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.321605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.323024 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.323028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.323900 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.323911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.324917 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.324921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.325096 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.325812 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.325831 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.328068 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.332499 (+ 4431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.332503 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.332506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.332924 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.333395 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.333399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.333401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.333583 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.334239 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.334243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.334393 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.334519 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.334894 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 05:41:04.335325 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.320409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.30.137.253:41433)
0809 05:41:04.320642 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.320649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.320672 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:41:04.320744 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.320750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.320832 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.320941 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.321760 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.322877 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.324067 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.324765 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.324897 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.328308 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.328315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.328321 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.328363 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.332323 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.333063 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.333070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.333074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.333251 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.333721 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.333726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.333729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.334116 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.334286 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.334733 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.335006 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 05:41:04.336597 455 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:41:04.337649 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.337240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33858)
0809 05:41:04.337420 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.337507 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 05:41:04.338591 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.338245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33874)
0809 05:41:04.338405 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.338484 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 05:41:04.338850 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.363215 571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 2
I20260809 05:41:04.364503 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.364002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33884)
0809 05:41:04.364209 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.364319 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 05:41:04.365787 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.365324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33898)
0809 05:41:04.365502 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.365600 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 05:41:04.366094 571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.390103 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 1
I20260809 05:41:04.391616 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.391001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33908)
0809 05:41:04.391264 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.391394 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 05:41:04.392920 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.392528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33920)
0809 05:41:04.392694 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.392787 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 05:41:04.393162 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.457365 571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 3
I20260809 05:41:04.458961 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.458311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33936)
0809 05:41:04.458556 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.458762 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 05:41:04.460363 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.459835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33938)
0809 05:41:04.460140 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.460218 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
W20260809 05:41:04.460678 571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.461417 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 2
I20260809 05:41:04.462783 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.462158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33950)
0809 05:41:04.462437 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.462553 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 05:41:04.464030 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.463564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33958)
0809 05:41:04.463762 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.463864 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 05:41:04.464313 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.570607 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 3
I20260809 05:41:04.572099 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.571441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33968)
0809 05:41:04.571704 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.571826 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 05:41:04.573390 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.572944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33984)
0809 05:41:04.573131 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.573247 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 05:41:04.573650 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.603941 571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 4
I20260809 05:41:04.605252 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.604725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:33992)
0809 05:41:04.604985 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.605087 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 05:41:04.606429 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.606071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:34006)
0809 05:41:04.606245 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.606313 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 05:41:04.606735 571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
W20260809 05:41:04.655084 582 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 05:41:04.655500 582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 05:41:04.655643 582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 05:41:04.655782 582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 05:41:04.663703 582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 05:41:04.663815 582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 05:41:04.663913 582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 05:41:04.664017 582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 05:41:04.664132 582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 05:41:04.674788 582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.137.212:46195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.137.254:37629,127.30.137.253:41433,127.30.137.252:42307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/krb5kdc/kudu_127.30.137.252.keytab
--principal=kudu/127.30.137.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.137.252:42307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.137.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:51:24 UTC on ccycloud.quasar-zwepce.root.comops.site
build id 4575769
ASAN enabled
I20260809 05:41:04.676213 582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 05:41:04.677616 582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 05:41:04.690002 593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:41:04.689971 591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 05:41:04.690024 589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 05:41:04.691290 582 server_base.cc:1034] running on GCE node
I20260809 05:41:04.692191 582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 05:41:04.693776 582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 05:41:04.695035 582 hybrid_clock.cc:648] HybridClock initialized: now 1786254064694951 us; error 50 us; skew 500 ppm
I20260809 05:41:04.695426 582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 05:41:04.699074 582 init.cc:377] Logged in from keytab as kudu/127.30.137.252@KRBTEST.COM (short username kudu)
I20260809 05:41:04.700729 582 webserver.cc:460] Webserver started at http://127.30.137.252:43207/ using document root <none> and password file <none>
I20260809 05:41:04.701484 582 fs_manager.cc:359] Metadata directory not provided
I20260809 05:41:04.701606 582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 05:41:04.701982 582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 05:41:04.705204 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data/instance:
uuid: "200747a29fab47f9964a4bc037f1f2c5"
format_stamp: "Formatted at 2026-08-09 05:41:04 on dist-test-slave-3kk6"
server_key: "ce6e1f396a1a6617a388ad60d97b630d"
server_key_iv: "0d4a27e21f90879e493efd92aff85b79"
server_key_version: "encryptionkey@0"
I20260809 05:41:04.706081 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal/instance:
uuid: "200747a29fab47f9964a4bc037f1f2c5"
format_stamp: "Formatted at 2026-08-09 05:41:04 on dist-test-slave-3kk6"
server_key: "ce6e1f396a1a6617a388ad60d97b630d"
server_key_iv: "0d4a27e21f90879e493efd92aff85b79"
server_key_version: "encryptionkey@0"
I20260809 05:41:04.711722 582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 05:41:04.715464 602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 05:41:04.716869 582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 05:41:04.717069 582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data,/tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal
uuid: "200747a29fab47f9964a4bc037f1f2c5"
format_stamp: "Formatted at 2026-08-09 05:41:04 on dist-test-slave-3kk6"
server_key: "ce6e1f396a1a6617a388ad60d97b630d"
server_key_iv: "0d4a27e21f90879e493efd92aff85b79"
server_key_version: "encryptionkey@0"
I20260809 05:41:04.717327 582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 05:41:04.745980 582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 05:41:04.750556 582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 05:41:04.750933 582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 05:41:04.750888 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 4
I20260809 05:41:04.752153 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.751553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:34010)
0809 05:41:04.751809 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.751933 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 05:41:04.753369 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.752934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:34014)
0809 05:41:04.753117 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.753208 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 05:41:04.753623 455 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.137.252:42307: Network error: Client connection negotiation failed: client connection to 127.30.137.252:42307: connect: Connection refused (error 111)
I20260809 05:41:04.771629 582 rpc_server.cc:307] RPC server started. Bound to: 127.30.137.252:42307
I20260809 05:41:04.771718 677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.137.252:42307 every 8 connection(s)
I20260809 05:41:04.773244 582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/data/info.pb
I20260809 05:41:04.777369 678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 05:41:04.780256 31271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu as pid 582
I20260809 05:41:04.780453 31271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUEwy2v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786253675866932-31271-0/minicluster-data/master-2/wal/instance
I20260809 05:41:04.780961 31271 external_mini_cluster.cc:1428] Setting key e444351340304c3d89a2874af3514927
I20260809 05:41:04.786420 678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:41:04.806887 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.789531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.30.137.254:37629)
0809 05:41:04.790397 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.790403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.790789 (+ 386us) server_negotiation.cc:407] Connection header received
0809 05:41:04.791063 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.791067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.791132 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.791262 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.792506 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.793622 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.795874 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.796302 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.796414 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.800348 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.800353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.800357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.800407 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.803704 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.804540 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.804543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.804546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.804709 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.805181 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.805186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.805189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.805557 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.805708 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.806213 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.806443 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":599,"threads_started":1}
I20260809 05:41:04.806916 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.789453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.254:37629 (local address 127.0.0.1:37316)
0809 05:41:04.790571 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.790620 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.790896 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.791335 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.791342 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.792077 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.792342 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.792351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.793882 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.793888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.795697 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.795714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.796582 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.796591 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.796796 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.797511 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.797534 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.800123 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.803876 (+ 3753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.803885 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.803890 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.804394 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.804844 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.804849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.804853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.805039 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.805702 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.805707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.805886 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.806032 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.806293 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":32,"thread_start_us":366,"threads_started":1}
I20260809 05:41:04.806995 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.783090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.30.137.252:42307)
0809 05:41:04.783973 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.783992 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.784045 (+ 53us) server_negotiation.cc:407] Connection header received
0809 05:41:04.786483 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.786515 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.787134 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.787640 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.789300 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.791366 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.792744 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.793791 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.793980 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.798937 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.798945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.798958 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.798994 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.801983 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.802798 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.802810 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.802820 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.803021 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.803422 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.803429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.803433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.804166 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.804419 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.805029 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.805490 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":618,"spinlock_wait_cycles":17792,"thread_start_us":343,"threads_started":1}
I20260809 05:41:04.809016 678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.254@KRBTEST.COM
I20260809 05:41:04.824698 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.809920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39380 (local address 127.30.137.253:41433)
0809 05:41:04.810164 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.810170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.810347 (+ 177us) server_negotiation.cc:407] Connection header received
0809 05:41:04.810532 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.810538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.810627 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.810780 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.811597 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.812723 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.814250 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.814983 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.815128 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.818509 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.818518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.818523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.818568 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.821924 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.822727 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.822733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.822737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.822909 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.823300 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.823306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.823309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.823692 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.823845 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.824217 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.824390 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 05:41:04.824961 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.809925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.253:41433 (local address 127.0.0.1:39380)
0809 05:41:04.810247 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.810269 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.810384 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.810798 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.810803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.811297 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.811444 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.811452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.812869 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.812875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.814098 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.814110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.815167 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.815173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.815397 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.816051 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.816076 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.818290 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.822104 (+ 3814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.822109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.822113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.822566 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.823037 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.823041 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.823043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.823186 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.823814 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.823818 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.823952 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.824115 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.824446 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260809 05:41:04.826332 678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } has no permanent_uuid. Determining permanent_uuid...
I20260809 05:41:04.827181 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.810607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.30.137.254:37629)
0809 05:41:04.810792 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.810798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.810816 (+ 18us) server_negotiation.cc:407] Connection header received
0809 05:41:04.810875 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.810882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.810968 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.811076 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.812598 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.813610 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.815425 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.816068 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.816233 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.820591 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.820596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.820599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.820632 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.823930 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.824762 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.824767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.824770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.824932 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.825436 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.825441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.825444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.825788 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.826119 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.826547 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.826835 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.252@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.137.253@KRBTEST.COM
I20260809 05:41:04.841048 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.827237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.30.137.252:42307)
0809 05:41:04.827383 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.827387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.827443 (+ 56us) server_negotiation.cc:407] Connection header received
0809 05:41:04.827634 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.827640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.827709 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.827758 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.828496 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.829407 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.830574 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.831076 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.831224 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.835519 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.835524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.835528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.835559 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.838231 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.839057 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.839066 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.839069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.839203 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.839574 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.839582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.839585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.839884 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.840043 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.840397 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.840648 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 05:41:04.841171 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.827164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:34032)
0809 05:41:04.827375 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.827394 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.827506 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.827888 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.827891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.828234 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.828361 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.828369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.829532 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.829539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.830425 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.830436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.831235 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.831240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.831484 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.832199 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.832224 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.835249 (+ 3025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.838384 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.838390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.838394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.838932 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.839327 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.839337 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.839339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.839466 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.840016 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.840021 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.840167 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.840321 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.840799 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":180}
I20260809 05:41:04.845865 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.829194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.30.137.253:41433)
0809 05:41:04.829367 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.829372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.829386 (+ 14us) server_negotiation.cc:407] Connection header received
0809 05:41:04.829429 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.829432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.829489 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.829601 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.830918 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.831771 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.833715 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.834478 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.834623 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.839763 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.839770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.839774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.839822 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.843021 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.843810 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.843817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.843822 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.843974 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.844386 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.844392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.844396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.844677 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.844963 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.845279 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.845530 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 05:41:04.846764 678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: Bootstrap starting.
I20260809 05:41:04.850921 678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: Neither blocks nor log segments found. Creating new log.
I20260809 05:41:04.851572 31271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 05:41:04.852232 678 log.cc:824] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: Log is configured to *not* fsync() on all Append() calls
I20260809 05:41:04.855285 678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: No bootstrap required, opened a new log
I20260809 05:41:04.854056 413 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ca9e2a1da2824caab42b39dd4d3f5799"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37328
I20260809 05:41:04.863087 678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:04.863533 678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 200747a29fab47f9964a4bc037f1f2c5, State: Initialized, Role: FOLLOWER
I20260809 05:41:04.864050 678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:04.865208 686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:04.865653 686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:04.866324 678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 05:41:04.877117 705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:41:04.877297 705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:41:04.874980 582 master_runner.cc:186] Error getting master registration for 127.30.137.254:37629: OK, instance_id { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" instance_seqno: 1786254063606266 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:41:04.887189 582 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 05:41:04.905968 571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 5
Aug 09 05:41:04 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.253@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
I20260809 05:41:04.924225 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.907132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34048 (local address 127.30.137.252:42307)
0809 05:41:04.907343 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:04.907351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:04.907375 (+ 24us) server_negotiation.cc:407] Connection header received
0809 05:41:04.907450 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:04.907456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:04.907537 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:04.907665 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:04.908723 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.909929 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.911861 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.912691 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.912874 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.916948 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:04.916955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:04.916963 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:04.917016 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:04.920510 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.921611 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.921618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.921625 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.921879 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:04.922347 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:04.922352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:04.922355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:04.922872 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:04.923067 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:04.923526 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 05:41:04.923800 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 05:41:04.924257 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:04.906897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:34048)
0809 05:41:04.907118 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:04.907142 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:04.907294 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:04.907713 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:04.907720 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:04.908292 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:04.908521 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.908536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.910114 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.910119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.911630 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:04.911664 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:04.912890 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:04.912897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:04.913144 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:04.913968 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:04.913987 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:04.916697 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:04.920730 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.920738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.920744 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.921459 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.922019 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:04.922024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:04.922028 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:04.922220 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:04.923017 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:04.923023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:04.923201 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 05:41:04.923414 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 05:41:04.923760 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 05:41:04.929273 571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: Bootstrap starting.
I20260809 05:41:04.933624 571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: Neither blocks nor log segments found. Creating new log.
I20260809 05:41:04.935127 571 log.cc:824] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: Log is configured to *not* fsync() on all Append() calls
I20260809 05:41:04.938510 571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: No bootstrap required, opened a new log
I20260809 05:41:04.950603 571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:04.951200 571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e83fcbd087641c6abae8544b66a2ddd, State: Initialized, Role: FOLLOWER
I20260809 05:41:04.951785 571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:04.952942 710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:04.953444 710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:04.954360 571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: configured and running, proceeding with master startup.
W20260809 05:41:04.967850 729 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 05:41:04.968102 729 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 05:41:04.964545 468 master_runner.cc:186] Error getting master registration for 127.30.137.254:37629: OK, instance_id { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" instance_seqno: 1786254063606266 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 05:41:04.977613 468 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 05:41:05.003710 612 tablet.cc:2369] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:41:05.030891 455 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } attempt: 5
Aug 09 05:41:05 dist-test-slave-3kk6 krb5kdc[324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786254063, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.137.254@KRBTEST.COM for kudu/127.30.137.252@KRBTEST.COM
I20260809 05:41:05.051524 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:05.032884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.30.137.252:42307)
0809 05:41:05.033083 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.033091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.033114 (+ 23us) server_negotiation.cc:407] Connection header received
0809 05:41:05.033193 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.033198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.033308 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.033414 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.034739 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.035922 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.037864 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.038676 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.038830 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:05.043369 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:05.043377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:05.043382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:05.043429 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:05.047682 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.048622 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.048628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.048632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.048826 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.049296 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.049301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.049305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.049670 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:05.049827 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:05.050862 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0809 05:41:05.051124 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:41:05.052143 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:05.031838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.137.252:42307 (local address 127.0.0.1:34054)
0809 05:41:05.032328 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 05:41:05.032352 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 05:41:05.032496 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 05:41:05.033632 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 05:41:05.033637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 05:41:05.034233 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 05:41:05.034480 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:05.034493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.036097 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.036103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:05.037696 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 05:41:05.037713 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.038868 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.038875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 05:41:05.039081 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:05.039915 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 05:41:05.039942 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 05:41:05.043131 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 05:41:05.047875 (+ 4744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:05.047884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:05.047889 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:05.048491 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:05.048976 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 05:41:05.048982 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 05:41:05.048986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 05:41:05.049182 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 05:41:05.049814 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 05:41:05.049819 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 05:41:05.050012 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 05:41:05.050170 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 05:41:05.050492 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338}
I20260809 05:41:05.058898 455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799: Bootstrap starting.
I20260809 05:41:05.063380 455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799: Neither blocks nor log segments found. Creating new log.
I20260809 05:41:05.064836 455 log.cc:824] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799: Log is configured to *not* fsync() on all Append() calls
I20260809 05:41:05.075213 455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799: No bootstrap required, opened a new log
I20260809 05:41:05.083751 455 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:05.084209 455 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9e2a1da2824caab42b39dd4d3f5799, State: Initialized, Role: FOLLOWER
I20260809 05:41:05.084789 455 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:05.086195 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.086988 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:05.087401 455 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 05:41:05.087487 413 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 05:41:05.087704 413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:41:05.091354 413 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:05.094012 532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2e83fcbd087641c6abae8544b66a2ddd"
I20260809 05:41:05.094586 532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [term 0 FOLLOWER]: Advancing to term 1
W20260809 05:41:05.095392 381 tablet.cc:2369] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:41:05.098784 532 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca9e2a1da2824caab42b39dd4d3f5799 in term 1.
I20260809 05:41:05.099726 378 leader_election.cc:302] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e83fcbd087641c6abae8544b66a2ddd, ca9e2a1da2824caab42b39dd4d3f5799; no voters:
I20260809 05:41:05.100791 413 leader_election.cc:288] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [CANDIDATE]: Term 1 election: Requested vote from peers 2e83fcbd087641c6abae8544b66a2ddd (127.30.137.253:41433), 200747a29fab47f9964a4bc037f1f2c5 (127.30.137.252:42307)
I20260809 05:41:05.101329 734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 05:41:05.101547 640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "200747a29fab47f9964a4bc037f1f2c5"
I20260809 05:41:05.102106 640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 05:41:05.107420 640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca9e2a1da2824caab42b39dd4d3f5799 in term 1.
I20260809 05:41:05.108595 734 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [term 1 LEADER]: Becoming Leader. State: Replica: ca9e2a1da2824caab42b39dd4d3f5799, State: Running, Role: LEADER
I20260809 05:41:05.109251 734 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } }
I20260809 05:41:05.128582 736 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9e2a1da2824caab42b39dd4d3f5799. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.129066 736 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: This master's current role is: LEADER
I20260809 05:41:05.139300 757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 05:41:05.157582 501 tablet.cc:2369] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 05:41:05.162626 757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 05:41:05.166330 772 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 05:41:05.171739 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:05.133575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.30.137.253:41433)
0809 05:41:05.133790 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.133797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.139243 (+ 5446us) server_negotiation.cc:407] Connection header received
0809 05:41:05.139585 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.139591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.139693 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.139787 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.154812 (+ 15025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.155988 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.158110 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.158865 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.158968 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:05.163040 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:05.163047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:05.163053 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:05.163106 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:05.167441 (+ 4335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.168935 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.168943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.168948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.169117 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.169675 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.169683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.169687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.170111 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:05.170389 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:05.171020 (+ 631us) server_negotiation.cc:299] Negotiation successful
0809 05:41:05.171310 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 05:41:05.180868 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:05.139420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.30.137.252:42307)
0809 05:41:05.139620 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.139625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.152623 (+ 12998us) server_negotiation.cc:407] Connection header received
0809 05:41:05.153064 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.153070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.153181 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.153267 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.158948 (+ 5681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.159981 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.166763 (+ 6782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.167571 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.167699 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:05.171259 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:05.171267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:05.171272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:05.171316 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:05.177265 (+ 5949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.178192 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.178199 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.178202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.178371 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.178889 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.178894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.178898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.179315 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:05.179593 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:05.180176 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 05:41:05.180448 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 05:41:05.194753 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:05.134941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.30.137.254:37629)
0809 05:41:05.135141 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.135150 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.147805 (+ 12655us) server_negotiation.cc:407] Connection header received
0809 05:41:05.149494 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.149501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.149592 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.149669 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.154952 (+ 5283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.156064 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.163441 (+ 7377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:05.164320 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:05.164434 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:05.186898 (+ 22464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 05:41:05.186907 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 05:41:05.186912 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 05:41:05.186960 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 05:41:05.191005 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.192050 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.192056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.192060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.192270 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 05:41:05.192778 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 05:41:05.192783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 05:41:05.192787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 05:41:05.193237 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 05:41:05.193513 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:05.194104 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 05:41:05.194333 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 05:41:05.213481 640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [term 1 FOLLOWER]: Refusing update from remote peer ca9e2a1da2824caab42b39dd4d3f5799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:41:05.215220 736 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:41:05.226876 532 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [term 1 FOLLOWER]: Refusing update from remote peer ca9e2a1da2824caab42b39dd4d3f5799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 05:41:05.228389 736 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 05:41:05.254122 686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9e2a1da2824caab42b39dd4d3f5799. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.254503 686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:05.256629 710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9e2a1da2824caab42b39dd4d3f5799. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.256971 710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:05.271549 773 mvcc.cc:205] Tried to move back new op lower bound from 7316496651079725056 to 7316496650693808128. Current Snapshot: MvccSnapshot[applied={T|T < 7316496651079725056}]
I20260809 05:41:05.283308 686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.283620 686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:05.294085 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.294692 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: This master's current role is: LEADER
I20260809 05:41:05.295609 757 catalog_manager.cc:1347] Generated new cluster ID: f6f01506edc049f4a734a30d896b5d3e
I20260809 05:41:05.295717 757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 05:41:05.315188 775 mvcc.cc:205] Tried to move back new op lower bound from 7316496651079725056 to 7316496650693808128. Current Snapshot: MvccSnapshot[applied={T|T < 7316496651079725056}]
I20260809 05:41:05.326531 778 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.329071 778 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799 [sys.catalog]: This master's current role is: LEADER
I20260809 05:41:05.333721 757 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 05:41:05.334897 710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca9e2a1da2824caab42b39dd4d3f5799" member_type: VOTER last_known_addr { host: "127.30.137.254" port: 37629 } } peers { permanent_uuid: "2e83fcbd087641c6abae8544b66a2ddd" member_type: VOTER last_known_addr { host: "127.30.137.253" port: 41433 } } peers { permanent_uuid: "200747a29fab47f9964a4bc037f1f2c5" member_type: VOTER last_known_addr { host: "127.30.137.252" port: 42307 } } }
I20260809 05:41:05.335170 710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 05:41:05.336076 757 catalog_manager.cc:1504] Loading token signing keys...
I20260809 05:41:05.362501 757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca9e2a1da2824caab42b39dd4d3f5799: Generated new TSK 0
I20260809 05:41:05.363755 757 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 05:41:05.486508 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.481116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.30.137.253:41433)
0809 05:41:05.481333 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.481339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.483159 (+ 1820us) server_negotiation.cc:407] Connection header received
0809 05:41:05.484275 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.484283 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.484386 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.484482 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.486227 (+ 1745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39406: BlockingRecv error: recv got EOF from 127.0.0.1:39406 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 05:41:05.490495 684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.480772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.30.137.254:37629)
0809 05:41:05.481027 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.481034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.488168 (+ 7134us) server_negotiation.cc:407] Connection header received
0809 05:41:05.488936 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.488942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.489037 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.489120 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.490213 (+ 1093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37348: BlockingRecv error: recv got EOF from 127.0.0.1:37348 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 05:41:05.494500 682 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.483733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.30.137.252:42307)
0809 05:41:05.483947 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.483956 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.492080 (+ 8124us) server_negotiation.cc:407] Connection header received
0809 05:41:05.492922 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.492929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.493024 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.493100 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.494235 (+ 1135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34072: BlockingRecv error: recv got EOF from 127.0.0.1:34072 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 05:41:05.501719 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.498957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.30.137.253:41433)
0809 05:41:05.499171 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.499177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.499816 (+ 639us) server_negotiation.cc:407] Connection header received
0809 05:41:05.500094 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.500101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.500251 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.500380 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.501378 (+ 998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39414: BlockingRecv error: recv got EOF from 127.0.0.1:39414 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 05:41:05.508896 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.506680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.30.137.253:41433)
0809 05:41:05.506863 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.506874 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.507278 (+ 404us) server_negotiation.cc:407] Connection header received
0809 05:41:05.507517 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.507524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.507615 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.507733 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.508658 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39424: BlockingRecv error: recv got EOF from 127.0.0.1:39424 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 05:41:05.519531 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.516887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.30.137.253:41433)
0809 05:41:05.517087 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.517093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.517701 (+ 608us) server_negotiation.cc:407] Connection header received
0809 05:41:05.517938 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.517944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.518063 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.518193 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.519222 (+ 1029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39434: BlockingRecv error: recv got EOF from 127.0.0.1:39434 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 05:41:05.529764 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.527377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.30.137.253:41433)
0809 05:41:05.527624 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.527630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.528148 (+ 518us) server_negotiation.cc:407] Connection header received
0809 05:41:05.528347 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.528353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.528453 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.528598 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.529491 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39444: BlockingRecv error: recv got EOF from 127.0.0.1:39444 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260809 05:41:05.552301 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.549615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.30.137.253:41433)
0809 05:41:05.549823 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.549829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.550496 (+ 667us) server_negotiation.cc:407] Connection header received
0809 05:41:05.550754 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.550760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.550873 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.551003 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.552038 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39452: BlockingRecv error: recv got EOF from 127.0.0.1:39452 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 05:41:05.575096 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.572314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.30.137.253:41433)
0809 05:41:05.572508 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.572515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.573055 (+ 540us) server_negotiation.cc:407] Connection header received
0809 05:41:05.573495 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.573501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.573625 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.573760 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.574837 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39466: BlockingRecv error: recv got EOF from 127.0.0.1:39466 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 05:41:05.613327 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.610850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.30.137.253:41433)
0809 05:41:05.611065 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.611071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.611404 (+ 333us) server_negotiation.cc:407] Connection header received
0809 05:41:05.611673 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.611681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.611800 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.611937 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.612996 (+ 1059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39474: BlockingRecv error: recv got EOF from 127.0.0.1:39474 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 05:41:05.683553 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.681318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.30.137.253:41433)
0809 05:41:05.681550 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.681558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.681969 (+ 411us) server_negotiation.cc:407] Connection header received
0809 05:41:05.682162 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.682168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.682282 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.682412 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.683334 (+ 922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39490: BlockingRecv error: recv got EOF from 127.0.0.1:39490 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 05:41:05.817469 579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 05:41:05.815568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.30.137.253:41433)
0809 05:41:05.815769 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:05.815773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:05.816033 (+ 260us) server_negotiation.cc:407] Connection header received
0809 05:41:05.816216 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:05.816220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:05.816297 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:05.816399 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 05:41:05.817213 (+ 814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39504: BlockingRecv error: recv got EOF from 127.0.0.1:39504 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 05:41:05.879627 705 catalog_manager.cc:1259] Loaded cluster ID: f6f01506edc049f4a734a30d896b5d3e
I20260809 05:41:05.879798 705 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: loading cluster ID for follower catalog manager: success
I20260809 05:41:05.882407 705 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: acquiring CA information for follower catalog manager: success
I20260809 05:41:05.884328 705 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 200747a29fab47f9964a4bc037f1f2c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:41:05.970745 729 catalog_manager.cc:1259] Loaded cluster ID: f6f01506edc049f4a734a30d896b5d3e
I20260809 05:41:05.970854 729 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: loading cluster ID for follower catalog manager: success
I20260809 05:41:05.973619 729 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: acquiring CA information for follower catalog manager: success
I20260809 05:41:05.975596 729 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e83fcbd087641c6abae8544b66a2ddd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 05:41:06.083860 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:06.077728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.30.137.253:41433)
0809 05:41:06.077994 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:06.078002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:06.078242 (+ 240us) server_negotiation.cc:407] Connection header received
0809 05:41:06.078500 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:06.078504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:06.078698 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:06.078837 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:06.079329 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.080054 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.081494 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.081909 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.082057 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:06.082609 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:06.083213 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:06.083399 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:06.083467 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 05:41:06.083558 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 05:41:06.098867 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:06.090947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.30.137.252:42307)
0809 05:41:06.091511 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:06.091516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:06.091595 (+ 79us) server_negotiation.cc:407] Connection header received
0809 05:41:06.091817 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:06.091822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:06.092000 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:06.092162 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:06.092610 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.093354 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.095180 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.095950 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.096112 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:06.097008 (+ 896us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:06.097911 (+ 903us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:06.098071 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:06.098325 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 05:41:06.098443 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":280,"threads_started":1}
I20260809 05:41:06.117120 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:06.108044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37362 (local address 127.30.137.254:37629)
0809 05:41:06.109885 (+ 1841us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:06.109895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:06.109923 (+ 28us) server_negotiation.cc:407] Connection header received
0809 05:41:06.110094 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:06.110104 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:06.110419 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:06.110719 (+ 300us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:06.111529 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.112524 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.114289 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.114827 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.115596 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:06.115798 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:06.116321 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:06.116494 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:06.116645 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 05:41:06.116739 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":1492,"threads_started":1}
I20260809 05:41:06.119904 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:06.108130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39524 (local address 127.30.137.253:41433)
0809 05:41:06.108383 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:06.108393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:06.111639 (+ 3246us) server_negotiation.cc:407] Connection header received
0809 05:41:06.111932 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:06.111939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:06.112048 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:06.112140 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:06.113414 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.115035 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.117218 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.117873 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.118030 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:06.118709 (+ 679us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:06.118845 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:06.118957 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:06.119464 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 05:41:06.119549 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 05:41:06.121874 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 05:41:06.111921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.30.137.252:42307)
0809 05:41:06.112110 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 05:41:06.112116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 05:41:06.115426 (+ 3310us) server_negotiation.cc:407] Connection header received
0809 05:41:06.115575 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 05:41:06.115582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 05:41:06.115685 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 05:41:06.115798 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 05:41:06.116463 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.117689 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.119844 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 05:41:06.120438 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 05:41:06.120585 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 05:41:06.121142 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 05:41:06.121245 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 05:41:06.121353 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 05:41:06.121512 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 05:41:06.121598 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 05:41:06.123081 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 348
I20260809 05:41:06.124668 438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:41:06.244176 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 348
W20260809 05:41:06.276381 604 connection.cc:535] server connection from 127.0.0.1:34054 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 05:41:06.276800 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 468
I20260809 05:41:06.278040 560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:41:06.390758 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 468
I20260809 05:41:06.417560 31271 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu pid 582
I20260809 05:41:06.418885 672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 05:41:06.530946 31271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUEwy2v/build/asan/bin/kudu with pid 582
2026-08-09T05:41:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5189 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10689 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (390635 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST